-- Logs begin at Thu 2022-09-22 14:15:12 UTC, end at Thu 2022-09-22 15:22:03 UTC. -- Sep 22 14:43:24.224399 dv-cinder-844563-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 22 14:43:24.231679 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 22 14:43:24.233695 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 22 14:43:24 2022] *** Sep 22 14:43:24.233695 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 22 14:43:24.233695 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 22 14:43:24.233695 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: nodename: dv-cinder-844563-6-iscsi Sep 22 14:43:24.233695 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: machine: x86_64 Sep 22 14:43:24.233695 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: clock source: unix Sep 22 14:43:24.233695 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: pcre jit disabled Sep 22 14:43:24.234149 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: detected number of CPU cores: 4 Sep 22 14:43:24.234149 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: current working directory: / Sep 22 14:43:24.234149 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: detected binary path: /usr/bin/uwsgi-core Sep 22 14:43:24.234149 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: your processes number limit is 47680 Sep 22 14:43:24.234149 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: your memory page size is 4096 bytes Sep 22 14:43:24.234149 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: detected max file descriptor number: 2048 Sep 22 14:43:24.234149 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: lock engine: pthread robust mutexes Sep 22 14:43:24.234149 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: thunder lock: enabled Sep 22 14:43:24.234149 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 22 14:43:24.234149 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 22 14:43:24.275206 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: Python main interpreter initialized at 0x55aea5e419c0 Sep 22 14:43:24.275206 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: python threads support enabled Sep 22 14:43:24.275206 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: your server socket listen backlog is limited to 100 connections Sep 22 14:43:24.275206 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: your mercy for graceful operations on workers is 90 seconds Sep 22 14:43:24.275606 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: mapped 403077 bytes (393 KB) for 2 cores Sep 22 14:43:24.275688 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: *** Operational MODE: preforking *** Sep 22 14:43:24.275940 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: *** uWSGI is running in multiple interpreter mode *** Sep 22 14:43:24.275940 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: spawned uWSGI master process (pid: 81440) Sep 22 14:43:24.276151 dv-cinder-844563-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 22 14:43:24.277233 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: spawned uWSGI worker 1 (pid: 81454, cores: 1) Sep 22 14:43:24.277233 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: spawned uWSGI worker 2 (pid: 81455, cores: 1) Sep 22 14:43:24.277233 dv-cinder-844563-6-iscsi devstack@keystone.service[81440]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 22 14:43:28.562037 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 14:43:28.562037 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: from cryptography.utils import int_from_bytes Sep 22 14:43:28.562892 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 14:43:28.562892 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: from cryptography.utils import int_from_bytes Sep 22 14:43:28.583307 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 14:43:28.583307 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: from cryptography.utils import int_from_bytes Sep 22 14:43:28.583778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 14:43:28.583778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: from cryptography.utils import int_from_bytes Sep 22 14:43:28.613049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 22 14:43:28.613242 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 22 14:43:28.613412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] command line args: [] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 22 14:43:28.613682 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 22 14:43:28.613910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ================================================================================ {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 22 14:43:28.614108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] admin_token = **** {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.614353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] config_dir = [] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.614536 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.614715 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] config_source = [] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.614916 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] debug = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.615112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.615475 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.615653 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.615903 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.616150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.616380 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] list_limit = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.616556 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] log_config_append = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.616729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.616904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] log_dir = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.617084 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] log_file = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.617319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.617565 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.617817 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.618059 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.618233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.618413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.618639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.618920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.619139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.619370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.619600 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.619831 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.620084 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.620337 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.620607 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.621088 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.621273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] publish_errors = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.621495 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.621731 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.621909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.622151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.622384 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.622619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] standard_threads = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.622993 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.623304 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.623485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.623660 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] use_journal = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.623835 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] use_json = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.624010 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] use_stderr = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.624185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] use_syslog = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.624416 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.624676 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.624917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.625180 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.625420 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.625669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.625905 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.626154 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.626390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.626638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.626913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.627176 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.627412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.628203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.628436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.628679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.628934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.629168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.629421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.629676 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.629923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.630158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.630413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.630679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.630957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.631206 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.631636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.631871 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.632102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.632336 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.632583 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.632830 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.633060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.633289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.633542 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.633786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.634022 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.634265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.634496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.634766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.634986 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.635214 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.635441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.635671 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.635897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.636133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.636376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.636605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.636840 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.637067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.637307 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.637541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.637786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.638013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.638245 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.638469 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.638703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.638958 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.639191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.639419 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.639723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.639959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.640196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.640427 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.640687 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.640924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.641155 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.641389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.641625 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.641897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.642134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.642441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.642696 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.642956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.643806 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.644060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.644310 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.644546 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.644780 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.645021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.645258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.645549 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.645787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.646021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.646255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.646487 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.646719 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.647069 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.647332 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.647542 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.647775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.648030 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.648249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.648483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.648714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.648945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.649172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.649400 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.649626 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.649857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.650142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.650381 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.650613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.650882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.650975 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 22 14:43:28.651081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 22 14:43:28.651171 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.651284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] command line args: [] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 22 14:43:28.651364 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.651729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.651806 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 22 14:43:28.651973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.652059 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ================================================================================ {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 22 14:43:28.652206 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] auth.external = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.652421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] admin_token = **** {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.652499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.652717 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.652806 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] config_dir = [] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.652951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.653087 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.653172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] auth.password = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.653372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] config_source = [] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.653457 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] auth.token = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.653622 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.653707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] debug = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.653857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.653936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.654103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.654309 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.654388 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.654565 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.654649 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.654839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.655015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.655177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.655294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] credential.caching = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.655413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.655541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.655717 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] list_limit = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.655800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.655901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] log_config_append = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.656022 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.656107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.656247 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] log_dir = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.656325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.656425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] log_file = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.656511 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.656694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.656778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.656948 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.657024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.657194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.657282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.657429 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.657507 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.657608 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.657687 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.657847 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.657962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.658193 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.658297 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.658455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.658594 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.658713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.658946 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.659063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.659191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.659270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] federation.caching = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.659430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.659521 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.659667 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.659750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.659921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.660002 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.660151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.660241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.660398 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.660478 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.660635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.660886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.661023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.661115 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.661302 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] publish_errors = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.661424 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.661583 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.661668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.661829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.661976 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.662055 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] identity.caching = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.662159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.662289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.662414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.662544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.662653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.662777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.662937 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] standard_threads = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.663060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.663183 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.663275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.663431 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.663511 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.663609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.663689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.663787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] use_journal = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.663897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.664021 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] use_json = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.664145 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.664225 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] use_stderr = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.664376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.664488 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] use_syslog = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.664603 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.664832 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.664909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 14:43:28.665073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.665189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.665319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.665436 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.665562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.665702 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.665815 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.665947 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.666060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.666202 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.666330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.666442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.666565 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.666694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.666823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.666960 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.667042 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.667212 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.667291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.667463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.667542 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.667713 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.667799 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.667949 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.668028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.668186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.668278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.668426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.668513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.668652 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.668730 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.668883 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.668959 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.669130 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.669228 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.669364 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.669444 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.669606 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.669685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.669850 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.669943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.670079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.670158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.670313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.670392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.670541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.670631 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.670797 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.670881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.671034 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.671113 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.671274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.671389 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.671509 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.671599 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.671738 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.671817 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.671976 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.672057 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.672226 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.672351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.672475 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.672585 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.672708 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.672818 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.672939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.673068 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.673186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.673316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.673435 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.673552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.673673 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.673784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.673922 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.674014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.674157 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.674238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.674424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.674512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.674664 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.674758 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.674920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.675001 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.675152 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.675247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.675383 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.675464 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.675614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.675694 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.675852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.675933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.676101 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.676182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.676333 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.676413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.676563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.676644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.676790 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.676869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.677017 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.677095 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.677250 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.677330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.677496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.677575 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.677724 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.677811 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.677961 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.678043 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.678202 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.678282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.678428 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.678507 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.678661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.678788 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.678918 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.678999 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.679146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.679225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.679376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.679463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.679610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.679690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.679845 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.679923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.680078 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.680160 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.680310 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.680391 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.680561 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.680641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.680791 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.680872 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.681024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.681109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.681257 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.681335 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.681478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.681554 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.681711 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.681792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.681956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.682037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.682194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.682276 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.682424 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.682504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.682658 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.682755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.682918 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.683000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] resource.caching = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.683163 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.683242 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.683401 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.683480 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.683633 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.683713 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.683874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.683954 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.684112 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.684198 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.684343 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.684423 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.684584 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.684663 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.684816 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.684895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.685050 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.685129 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.685281 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] role.caching = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.685359 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.685515 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.685594 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] role.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.685744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.685824 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.685988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.686067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.686233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.686313 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.686472 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.686552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.686710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.686810 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.686976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.687057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.687213 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.687294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.687451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.687531 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.687689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.687769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.687917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.687997 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.688152 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.688237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.688384 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.688464 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.688620 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.688702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.688906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.688998 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.689073 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.689270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.689360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.689503 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.689590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.689749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.689830 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.690498 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.690587 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.690769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.690856 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.691018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.691098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.691248 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] auth.external = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.691329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.691479 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.691560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.691730 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.691809 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.691966 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.692056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.692194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.692275 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] auth.password = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.692422 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.692501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] auth.token = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.692650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.692731 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.692886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.692964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.693134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.693289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.693387 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.693461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.693598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.693678 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.693830 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.693909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.694057 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.694139 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.694291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] credential.caching = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.694370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] token.caching = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.694521 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.694600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.694787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.694873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.695025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.695106 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.695280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.695360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.695523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.695605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.695758 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.695836 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.695999 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.696069 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.696240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.696322 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.696478 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.696556 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.696709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.696789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.696945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.697026 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.697177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.697256 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.697419 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.697499 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.697700 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.697781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.697930 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.698018 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.698176 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.698257 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] federation.caching = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.698410 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.698497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.698663 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.698762 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.connection = **** {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.698935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.699015 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.699167 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.699258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.699403 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.699483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.699639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.699719 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.699870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.699957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.700102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.700182 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.700333 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.700413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.700566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.700655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.700804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.700884 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.701035 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.701115 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] identity.caching = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.701275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.701362 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.701506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.701589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.701749 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.701828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.701979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.702066 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.702218 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.702298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.702437 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.702555 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.702669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.702861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.702940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.703012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.703083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81454) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 22 14:43:28.703182 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.703417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.703651 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.703884 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.704130 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.704378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.704628 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.704885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.705135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.705408 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.705692 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.706445 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.706680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.706944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.707178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.707414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.707657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.707898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.708149 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.708382 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.708638 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.708886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.709124 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.709372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.709620 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.709853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.710090 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.710322 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.710560 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.710822 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.711069 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.711307 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.711543 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.711794 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.712042 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.712274 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.712505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.712753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.713001 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.713241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.713476 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.713714 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.713944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.714184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.714444 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.714677 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.714949 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.715198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.715446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.715682 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.715912 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.716148 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.716385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.716620 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.716851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.717098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.717346 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.717593 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.717839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.718086 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.718315 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.718559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.718819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.719056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.719290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.719790 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.720035 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.720398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.720639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.720896 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.721127 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.721412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.721651 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.721890 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.722130 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.722380 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.722626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.722891 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.723130 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.723374 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] resource.caching = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.723627 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.723879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.724117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.724368 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.724617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.724855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.725103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.725340 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.725579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.725817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] role.caching = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.726065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] role.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.726297 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.726545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.726819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.727073 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.727324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.727574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.727823 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.728072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.728322 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.728559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.728811 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.729062 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.729310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.729559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.729971 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.730209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.730461 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.730713 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.730991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.731242 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.731478 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.731735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.731988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.732229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.732462 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.732695 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.732836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:28.732920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.733066 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:28.733155 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.733287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:28.733408 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.733655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.733892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.734130 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.734361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.734598 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] token.caching = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.734861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.735112 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.735350 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.735605 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.735856 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.736095 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.736339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.736583 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.736830 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.737090 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.737331 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.737568 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.737818 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.738096 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.738330 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.738579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.738871 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.739136 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.connection = **** {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.739385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.739637 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.739876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.740120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.740358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.740596 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.740834 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.741071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.741316 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.741554 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.741634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:28.741801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.742038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.742288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.742526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.742795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.743032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.743274 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.743511 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 14:43:28.743681 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81455) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 22 14:43:28.769716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:28.769936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:28.770156 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:28.770387 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:28.770617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:28.778572 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:28.796776 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:28.797007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:28.824489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:28.835613 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:28.872758 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:28.872984 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:28.875920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:28.876158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.030537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.030771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.030859 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.030987 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.031099 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.031205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.031291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.031422 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.031508 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.031635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.031721 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.031909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.055336 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.055564 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.055694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.055798 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.055929 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.056014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.056151 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.056236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.056372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.056458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.056578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.056661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.056793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.057007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.067778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.067983 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.080293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.080546 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.081198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.081315 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.081434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.081521 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.081666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.081759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.081890 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.081974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.082096 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81454) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.082178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.082307 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.082519 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81455) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 14:43:29.099812 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.099989 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 14:43:29.100288 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.100486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 14:43:29.100625 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.100740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 14:43:29.100820 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.100999 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.101082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.101241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 14:43:29.101338 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.101494 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.101576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.101777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.102035 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.102291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.111053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.111329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.111598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.111899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.111983 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.112222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.112491 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.112773 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.118829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.119740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.121703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.121985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.122264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.122544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.122653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.122850 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.122953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.123113 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.123239 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.123379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.123528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.123649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.123817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.123900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.124080 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.124164 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.124356 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.124440 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.124637 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.124910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.125184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.125452 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.147660 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.148556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.149129 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.149413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.150035 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.150320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.154147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.154427 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.154708 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.154891 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.155177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.155574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.161700 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.161983 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.162459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.162769 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.166721 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.167034 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.167320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.167724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.168021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.168311 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.168600 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.168879 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.169156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.169419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.169518 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.169689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.169771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.169987 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.170280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.170562 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.185081 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.185358 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.186087 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.186374 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.189151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.190794 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.192874 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.193169 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.193888 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.194301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.197343 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.198536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.200420 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.201709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.203601 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.203895 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.204178 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.204461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.204739 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.204822 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.205019 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.205105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.205392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.206334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.206620 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.206944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.221389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.221676 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.221935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.222195 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.223009 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.223301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.223570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.223834 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.228943 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.230440 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.230566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.231960 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.232234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.232320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.233788 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.234064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.234933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.235220 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.235505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.235781 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.236169 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.236830 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.237156 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.237459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.237753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.238037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.247281 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.247568 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.247847 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.248122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.248499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.248780 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.249067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.249314 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.249599 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.249881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.250170 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.250453 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.250754 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.251025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.264932 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.266996 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.268958 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.271037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.271727 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.273231 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.273975 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.274696 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.275005 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.275412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.275581 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.277187 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.277470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.277754 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.281829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.283404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.284317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.285795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.286217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.286612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.286925 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.287211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.288690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.288982 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.289276 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.289552 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.294987 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.295283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.295572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.297537 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.297827 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.298111 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.302407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81454) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.302690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.303003 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.303280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.303574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.303835 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.304105 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.304378 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.304645 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.304954 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.305078 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81455) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 14:43:29.305293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.305412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.305564 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.305685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.305839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.305960 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.306112 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.306221 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81454) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.306388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.306679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.306992 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.307321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.307536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.307817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.308082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.308390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.308677 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81455) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 14:43:29.341110 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55aea5e419c0 pid: 81454 (default app) Sep 22 14:43:29.343705 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55aea5e419c0 pid: 81455 (default app) Sep 22 14:43:29.344739 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518baca6-5be4-426e-bcdb-8f818764130c None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:29.345030 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518baca6-5be4-426e-bcdb-8f818764130c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:29.345299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518baca6-5be4-426e-bcdb-8f818764130c None None] PATH_INFO: `/v3` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:29.349472 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Thu Sep 22 14:43:29 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:36.284088 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394a74e-5f77-4bf9-a96d-987f1911de5a None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:36.284709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394a74e-5f77-4bf9-a96d-987f1911de5a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:36.284709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c394a74e-5f77-4bf9-a96d-987f1911de5a None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:36.288927 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:36.294306 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a04e1-29a4-4578-8ff0-51c529244af0 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:36.294813 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a04e1-29a4-4578-8ff0-51c529244af0 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:36.294955 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a04e1-29a4-4578-8ff0-51c529244af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:36.366756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: INFO dbcounter [None req-f25a04e1-29a4-4578-8ff0-51c529244af0 None None] Registered counter for database keystone Sep 22 14:43:36.386651 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG oslo_db.sqlalchemy.engines [None req-f25a04e1-29a4-4578-8ff0-51c529244af0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81454) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 22 14:43:36.393254 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writer thread running {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 22 14:43:36.575473 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG passlib.handlers.bcrypt [None req-f25a04e1-29a4-4578-8ff0-51c529244af0 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81454) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 22 14:43:36.575874 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG passlib.handlers.bcrypt [None req-f25a04e1-29a4-4578-8ff0-51c529244af0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81454) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 22 14:43:36.633970 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-f25a04e1-29a4-4578-8ff0-51c529244af0 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:36.635618 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f25a04e1-29a4-4578-8ff0-51c529244af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:36.763168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 22 14:43:37.225124 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394a74e-5f77-4bf9-a96d-987f1911de5a None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:37.293606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: INFO dbcounter [None req-c394a74e-5f77-4bf9-a96d-987f1911de5a None None] Registered counter for database keystone Sep 22 14:43:37.314695 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG oslo_db.sqlalchemy.engines [None req-c394a74e-5f77-4bf9-a96d-987f1911de5a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81455) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 22 14:43:37.321063 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writer thread running {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 22 14:43:37.438405 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c394a74e-5f77-4bf9-a96d-987f1911de5a None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:37.443189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d19094-3589-4ae5-bf40-afe86ae6639d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:37.443811 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d19094-3589-4ae5-bf40-afe86ae6639d admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:37.444090 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d19094-3589-4ae5-bf40-afe86ae6639d admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:37.444354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d19094-3589-4ae5-bf40-afe86ae6639d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:37.452475 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d19094-3589-4ae5-bf40-afe86ae6639d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:37.504931 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d5d19094-3589-4ae5-bf40-afe86ae6639d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 22 14:43:37.505209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d19094-3589-4ae5-bf40-afe86ae6639d admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:37.514714 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Thu Sep 22 14:43:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 22 14:43:37.521207 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a04e1-29a4-4578-8ff0-51c529244af0 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:37.537950 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a04e1-29a4-4578-8ff0-51c529244af0 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:37.542391 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2de395-60c5-4e3d-86f7-0ad3b9a88b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:37.543529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2de395-60c5-4e3d-86f7-0ad3b9a88b3f admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:37.543810 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2de395-60c5-4e3d-86f7-0ad3b9a88b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:37.544088 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2de395-60c5-4e3d-86f7-0ad3b9a88b3f admin admin] PATH_INFO: `/v3/projects/82e4fd5adbe54235a872491a4adc6897` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:37.545661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2de395-60c5-4e3d-86f7-0ad3b9a88b3f admin admin] RBAC: Authorizing `identity:get_project(project_id=82e4fd5adbe54235a872491a4adc6897)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:37.599949 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ab2de395-60c5-4e3d-86f7-0ad3b9a88b3f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 22 14:43:37.600291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2de395-60c5-4e3d-86f7-0ad3b9a88b3f admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:37.604892 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Thu Sep 22 14:43:37 2022] GET /identity/v3/projects/82e4fd5adbe54235a872491a4adc6897 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:39.787763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e70ab-bc2d-4b67-8328-90e62503d748 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:39.788271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e70ab-bc2d-4b67-8328-90e62503d748 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:39.788271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e70ab-bc2d-4b67-8328-90e62503d748 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:39.792284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:43:39.797953 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6e310-e34a-4ec8-8ed9-04bdbf2a7d13 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:39.798379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6e310-e34a-4ec8-8ed9-04bdbf2a7d13 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:39.798507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6e310-e34a-4ec8-8ed9-04bdbf2a7d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:39.818940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-3cc6e310-e34a-4ec8-8ed9-04bdbf2a7d13 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:39.820478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3cc6e310-e34a-4ec8-8ed9-04bdbf2a7d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:39.828442 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 22 14:43:40.098263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e70ab-bc2d-4b67-8328-90e62503d748 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:40.110672 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e70ab-bc2d-4b67-8328-90e62503d748 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:40.115051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3e66d-6b03-4f8b-b6dd-485e49b10338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:40.115656 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3e66d-6b03-4f8b-b6dd-485e49b10338 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:40.115936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3e66d-6b03-4f8b-b6dd-485e49b10338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:40.116208 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3e66d-6b03-4f8b-b6dd-485e49b10338 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:40.123846 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3e66d-6b03-4f8b-b6dd-485e49b10338 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:40.125629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3e66d-6b03-4f8b-b6dd-485e49b10338 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:40.134389 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Thu Sep 22 14:43:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 14:43:40.140003 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6e310-e34a-4ec8-8ed9-04bdbf2a7d13 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:40.149562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6e310-e34a-4ec8-8ed9-04bdbf2a7d13 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:40.153976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a25615-770b-4cb7-b014-d5392f29a6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:40.154603 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a25615-770b-4cb7-b014-d5392f29a6b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:40.154918 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a25615-770b-4cb7-b014-d5392f29a6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:40.155193 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a25615-770b-4cb7-b014-d5392f29a6b2 admin admin] PATH_INFO: `/v3/users/4c56fbc6ee754a96af39128dae8f43a4` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:40.156497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a25615-770b-4cb7-b014-d5392f29a6b2 admin admin] RBAC: Authorizing `identity:get_user(user_id=4c56fbc6ee754a96af39128dae8f43a4)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:40.158331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a25615-770b-4cb7-b014-d5392f29a6b2 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:40.160428 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Thu Sep 22 14:43:40 2022] GET /identity/v3/users/4c56fbc6ee754a96af39128dae8f43a4 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:42.377432 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae8a85-bf60-4de5-99f9-c7bd8d86ead7 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:42.377991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae8a85-bf60-4de5-99f9-c7bd8d86ead7 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:42.377991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae8a85-bf60-4de5-99f9-c7bd8d86ead7 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:42.379355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:43:42.385412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcf334-7f04-4f0d-a088-87afc436f4e2 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:42.385923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcf334-7f04-4f0d-a088-87afc436f4e2 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:42.386108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcf334-7f04-4f0d-a088-87afc436f4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:42.405170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-b7bcf334-7f04-4f0d-a088-87afc436f4e2 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:42.406373 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b7bcf334-7f04-4f0d-a088-87afc436f4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:42.492599 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 6/11] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:42.495444 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266b254-b57a-4b1d-8cfd-dd4f053d770a None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:42.495731 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266b254-b57a-4b1d-8cfd-dd4f053d770a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:42.496008 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266b254-b57a-4b1d-8cfd-dd4f053d770a None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:42.497320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 6/12] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:42.502340 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52e99f-cce0-4ee2-a262-5bf99157bccc None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:42.502684 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52e99f-cce0-4ee2-a262-5bf99157bccc None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:42.502934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52e99f-cce0-4ee2-a262-5bf99157bccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:42.522259 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-6e52e99f-cce0-4ee2-a262-5bf99157bccc None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:42.523513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-6e52e99f-cce0-4ee2-a262-5bf99157bccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:42.540469 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:42.593205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:43:42.761692 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266b254-b57a-4b1d-8cfd-dd4f053d770a None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:42.859207 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52e99f-cce0-4ee2-a262-5bf99157bccc None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:42.906819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266b254-b57a-4b1d-8cfd-dd4f053d770a None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:42.941762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab550999-97f4-4d80-a53d-09d333c48472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:42.942614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab550999-97f4-4d80-a53d-09d333c48472 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:42.942918 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab550999-97f4-4d80-a53d-09d333c48472 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:42.943191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab550999-97f4-4d80-a53d-09d333c48472 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:42.954312 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab550999-97f4-4d80-a53d-09d333c48472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:43:42.960928 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52e99f-cce0-4ee2-a262-5bf99157bccc None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:42.964870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-ab550999-97f4-4d80-a53d-09d333c48472 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:43:42.964870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:42.964870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:42.964870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:42.964870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:42.964870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:42.964870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:42.964870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:42.964870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:42.964870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:42.964870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:42.964870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:42.964870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:43:42.964870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:43:42.964870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:43:42.964870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:43:42.965801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:43:42.965801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:43:42.965801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:43:42.965801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:43:42.965801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:42.965801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:42.965801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:42.965801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:42.965801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:42.965801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:42.965801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:42.965801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:42.965801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:42.965801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:42.965801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:42.965801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:42.966629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:42.966629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:42.966629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:42.966629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:42.966629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:43:42.966629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:43:42.966629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:43:42.966629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:43:42.966629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:43:42.966629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:43:42.966629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:43:42.966629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:43:42.966629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:43:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:43:42.972506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab550999-97f4-4d80-a53d-09d333c48472 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:42.997014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79004f2-536d-4739-96b8-8b5bfb19ac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:42.997631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79004f2-536d-4739-96b8-8b5bfb19ac48 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:42.997908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79004f2-536d-4739-96b8-8b5bfb19ac48 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:42.998185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79004f2-536d-4739-96b8-8b5bfb19ac48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:43.007197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:43.020000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:43.067646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79004f2-536d-4739-96b8-8b5bfb19ac48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:43.069125 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79004f2-536d-4739-96b8-8b5bfb19ac48 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:43.075288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab550999-97f4-4d80-a53d-09d333c48472 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:43.110115 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af0c43-a78c-464a-9b6c-9cbd818d4395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:43.110659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af0c43-a78c-464a-9b6c-9cbd818d4395 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:43.110968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af0c43-a78c-464a-9b6c-9cbd818d4395 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:43.111245 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af0c43-a78c-464a-9b6c-9cbd818d4395 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:43.111915 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af0c43-a78c-464a-9b6c-9cbd818d4395 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:43.112221 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af0c43-a78c-464a-9b6c-9cbd818d4395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:43.113601 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af0c43-a78c-464a-9b6c-9cbd818d4395 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:43.125630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:43:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:43:43.131465 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af0c43-a78c-464a-9b6c-9cbd818d4395 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:43.166965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 8/16] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:43:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 14:43:43.172710 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79004f2-536d-4739-96b8-8b5bfb19ac48 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:43.224002 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:43.225319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:43.232037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af0c43-a78c-464a-9b6c-9cbd818d4395 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:43.265100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f838bb-5b72-4483-a42e-46b046d0852c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:43.265736 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f838bb-5b72-4483-a42e-46b046d0852c None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:43.266014 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f838bb-5b72-4483-a42e-46b046d0852c None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:43.266293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f838bb-5b72-4483-a42e-46b046d0852c None admin] PATH_INFO: `/v3/users/4c56fbc6ee754a96af39128dae8f43a4` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:43.267868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f838bb-5b72-4483-a42e-46b046d0852c None admin] RBAC: Authorizing `identity:get_user(user_id=4c56fbc6ee754a96af39128dae8f43a4)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:43.269831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f838bb-5b72-4483-a42e-46b046d0852c None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:43.270493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79004f2-536d-4739-96b8-8b5bfb19ac48 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:43.271437 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:43:43 2022] GET /identity/v3/users/4c56fbc6ee754a96af39128dae8f43a4 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:43:43.276851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f838bb-5b72-4483-a42e-46b046d0852c None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:43.305677 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed164bc-cb27-4cd5-847d-b6b2e7749ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:43.306309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed164bc-cb27-4cd5-847d-b6b2e7749ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:43.306581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed164bc-cb27-4cd5-847d-b6b2e7749ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:43.306878 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed164bc-cb27-4cd5-847d-b6b2e7749ec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:43.317373 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed164bc-cb27-4cd5-847d-b6b2e7749ec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:43.318883 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed164bc-cb27-4cd5-847d-b6b2e7749ec6 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:43.337707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-eed164bc-cb27-4cd5-847d-b6b2e7749ec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:43:43.337707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:43.337707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:43.337707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:43.337707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:43.337707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:43.337707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:43.337707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:43.337707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:43.337707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:43.337707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:43.337707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:43.337707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:43:43.337707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:43:43.337707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:43:43.337707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:43:43.338589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:43.338589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:43.338589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:43.338589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:43.338589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:43.338589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:43.338589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:43.338589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:43.338589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:43.338589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:43.338589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:43.338589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:43.338589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:43.338589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:43.338589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:43.338589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:43.338589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:43:43.339446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:43:43.339446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:43:43.339446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:43:43.339446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 9/18] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:43:43.344657 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed164bc-cb27-4cd5-847d-b6b2e7749ec6 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:43.381565 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f838bb-5b72-4483-a42e-46b046d0852c None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:43.417715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac4285-878b-4153-93ca-624692dd2ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:43.418345 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac4285-878b-4153-93ca-624692dd2ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:43.418616 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac4285-878b-4153-93ca-624692dd2ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:43.418920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac4285-878b-4153-93ca-624692dd2ed6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:43.420123 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac4285-878b-4153-93ca-624692dd2ed6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:43.421551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac4285-878b-4153-93ca-624692dd2ed6 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:43.423121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 10/19] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:43:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:43:43.429022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac4285-878b-4153-93ca-624692dd2ed6 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:43.446779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed164bc-cb27-4cd5-847d-b6b2e7749ec6 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:43.449297 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:43.460714 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:43.482549 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13879caa-aaf5-4034-9b6a-b3ba44d23968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:43.483195 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13879caa-aaf5-4034-9b6a-b3ba44d23968 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:43.483478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13879caa-aaf5-4034-9b6a-b3ba44d23968 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:43.483748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13879caa-aaf5-4034-9b6a-b3ba44d23968 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:43.484389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13879caa-aaf5-4034-9b6a-b3ba44d23968 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:43.484691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13879caa-aaf5-4034-9b6a-b3ba44d23968 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:43.486076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13879caa-aaf5-4034-9b6a-b3ba44d23968 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:43.503607 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 10/20] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:43:43.535401 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac4285-878b-4153-93ca-624692dd2ed6 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:43.571437 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f93b2c-2ee0-4653-8afd-861c047adaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:43.572079 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f93b2c-2ee0-4653-8afd-861c047adaa3 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:43.572353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f93b2c-2ee0-4653-8afd-861c047adaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:43.572623 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f93b2c-2ee0-4653-8afd-861c047adaa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:43.573356 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f93b2c-2ee0-4653-8afd-861c047adaa3 None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.domain.id=default, user.id=4c56fbc6ee754a96af39128dae8f43a4) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:43.573658 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f93b2c-2ee0-4653-8afd-861c047adaa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:43.575162 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f93b2c-2ee0-4653-8afd-861c047adaa3 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:43.584855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 11/21] 10.222.0.26 () {56 vars in 1322 bytes} [Thu Sep 22 14:43:43 2022] GET /identity/v3/role_assignments?user.id=4c56fbc6ee754a96af39128dae8f43a4&scope.domain.id=default&role.id=32c8a81e339a462e973e76bb528ef35a => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:43:46.088592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0286b3b-0813-4107-a4f0-48e9f415658d None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.089570 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0286b3b-0813-4107-a4f0-48e9f415658d None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.090050 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0286b3b-0813-4107-a4f0-48e9f415658d None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.091619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 11/22] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:46.098040 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9793e-034c-4868-98dd-01b51f171d94 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.098479 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9793e-034c-4868-98dd-01b51f171d94 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.098649 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9793e-034c-4868-98dd-01b51f171d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.122424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG passlib.handlers.bcrypt [None req-52e9793e-034c-4868-98dd-01b51f171d94 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81455) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 22 14:43:46.122851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG passlib.handlers.bcrypt [None req-52e9793e-034c-4868-98dd-01b51f171d94 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81455) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 22 14:43:46.183870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-52e9793e-034c-4868-98dd-01b51f171d94 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:46.185499 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-52e9793e-034c-4868-98dd-01b51f171d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:46.239193 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1e9d6-93e6-4409-a222-be4a486110c8 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.239480 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1e9d6-93e6-4409-a222-be4a486110c8 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.239739 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1e9d6-93e6-4409-a222-be4a486110c8 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.241108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 12/23] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:46.246850 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb7cb3-d521-42a5-b67b-a3aae0913b42 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.247147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb7cb3-d521-42a5-b67b-a3aae0913b42 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.247412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb7cb3-d521-42a5-b67b-a3aae0913b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.266945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-0acb7cb3-d521-42a5-b67b-a3aae0913b42 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:46.268776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0acb7cb3-d521-42a5-b67b-a3aae0913b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:46.336910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 12/24] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:46.347421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 13/25] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:46.362319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf862b-b7d7-4049-b7d5-1c5f06238345 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.362604 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf862b-b7d7-4049-b7d5-1c5f06238345 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.362906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf862b-b7d7-4049-b7d5-1c5f06238345 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.364263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 13/26] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:46.369718 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b30e6f-77b5-46ca-9756-5d45f1c68569 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.370006 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b30e6f-77b5-46ca-9756-5d45f1c68569 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.370283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b30e6f-77b5-46ca-9756-5d45f1c68569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.399376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-28b30e6f-77b5-46ca-9756-5d45f1c68569 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:46.400605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-28b30e6f-77b5-46ca-9756-5d45f1c68569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:46.498932 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 14/27] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:46.532721 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76fcbe4-ce9f-418d-b006-b68ca392d3d9 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.533323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76fcbe4-ce9f-418d-b006-b68ca392d3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.534332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76fcbe4-ce9f-418d-b006-b68ca392d3d9 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.536732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 14/28] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:46.546495 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05954d22-d402-481e-917b-6703a6643389 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.546819 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05954d22-d402-481e-917b-6703a6643389 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.547100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05954d22-d402-481e-917b-6703a6643389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.575112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-05954d22-d402-481e-917b-6703a6643389 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:46.576330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-05954d22-d402-481e-917b-6703a6643389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:46.603530 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:46.668791 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 15/29] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:46.791419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76fcbe4-ce9f-418d-b006-b68ca392d3d9 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:46.823746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05954d22-d402-481e-917b-6703a6643389 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:46.830777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:46.940757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76fcbe4-ce9f-418d-b006-b68ca392d3d9 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:46.978688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb45941-dbc3-48b4-9cf6-a3ba9acd70cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:46.979363 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb45941-dbc3-48b4-9cf6-a3ba9acd70cb None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:46.979645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb45941-dbc3-48b4-9cf6-a3ba9acd70cb None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:46.979919 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb45941-dbc3-48b4-9cf6-a3ba9acd70cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:46.993387 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb45941-dbc3-48b4-9cf6-a3ba9acd70cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:43:46.999219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-5eb45941-dbc3-48b4-9cf6-a3ba9acd70cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:43:46.999219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:46.999219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:46.999219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:46.999219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:46.999219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:46.999219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:46.999219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:46.999219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:46.999219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:46.999219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:46.999219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:46.999219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:43:46.999219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:43:46.999219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:43:46.999219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:43:47.000064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:43:47.000064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:43:47.000064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:43:47.000064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:43:47.000064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:47.000064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:47.000064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:47.000064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:47.000064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:47.000064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:47.000064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:47.000064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:47.000064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:47.000064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:47.000064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:47.000064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:47.001254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:47.001254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:47.001254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:47.001254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:47.001254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:43:47.001254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:43:47.001254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:43:47.001254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:43:47.001254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:43:47.001254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:43:47.001254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:43:47.001254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:43:47.001254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 15/30] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:43:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:43:47.001897 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb45941-dbc3-48b4-9cf6-a3ba9acd70cb None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:47.071533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05954d22-d402-481e-917b-6703a6643389 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:47.106152 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:47.113215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea94f3a-b4ac-4809-9663-d048f79d9ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:47.113859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea94f3a-b4ac-4809-9663-d048f79d9ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:47.114141 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea94f3a-b4ac-4809-9663-d048f79d9ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:47.114415 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea94f3a-b4ac-4809-9663-d048f79d9ea2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:47.124948 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb45941-dbc3-48b4-9cf6-a3ba9acd70cb None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:47.127598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea94f3a-b4ac-4809-9663-d048f79d9ea2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:47.129977 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea94f3a-b4ac-4809-9663-d048f79d9ea2 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:47.149698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-eea94f3a-b4ac-4809-9663-d048f79d9ea2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:43:47.149698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:47.149698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:47.149698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:47.149698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:47.149698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:47.149698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:47.149698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:47.149698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:47.149698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:47.149698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:47.149698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:47.149698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:43:47.149698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:43:47.149698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:43:47.149698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:43:47.150609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:47.150609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:47.150609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:47.150609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:47.150609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:47.150609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:47.150609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:47.150609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:47.150609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:47.150609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:47.150609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:47.150609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:47.150609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:47.150609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:47.150609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:47.150609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:47.150609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:43:47.151749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:43:47.151749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:43:47.151749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:43:47.151749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 16/31] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:43:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:43:47.156877 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea94f3a-b4ac-4809-9663-d048f79d9ea2 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:47.169980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3048c58-a897-422a-846e-cb2312f3e751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:47.170342 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:47.170650 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3048c58-a897-422a-846e-cb2312f3e751 None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:47.170963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3048c58-a897-422a-846e-cb2312f3e751 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:47.171583 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3048c58-a897-422a-846e-cb2312f3e751 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:47.172028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3048c58-a897-422a-846e-cb2312f3e751 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:47.173736 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3048c58-a897-422a-846e-cb2312f3e751 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:47.291192 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea94f3a-b4ac-4809-9663-d048f79d9ea2 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:47.311809 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 16/32] 10.222.0.26 () {60 vars in 1131 bytes} [Thu Sep 22 14:43:47 2022] POST /identity/v3/roles => generated 219 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:43:47.313182 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3048c58-a897-422a-846e-cb2312f3e751 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:47.363247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed722267-09fe-4189-8b65-026dbf1edea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:47.364641 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed722267-09fe-4189-8b65-026dbf1edea5 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:47.366697 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed722267-09fe-4189-8b65-026dbf1edea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:47.367247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed722267-09fe-4189-8b65-026dbf1edea5 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:47.368166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed722267-09fe-4189-8b65-026dbf1edea5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:47.370001 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed722267-09fe-4189-8b65-026dbf1edea5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:47.371788 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed722267-09fe-4189-8b65-026dbf1edea5 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:47.398952 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 17/33] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:43:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:47.400176 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed722267-09fe-4189-8b65-026dbf1edea5 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:47.495782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3048c58-a897-422a-846e-cb2312f3e751 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:47.519169 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:47.548689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bd360-3139-4161-a4d2-d54cea3b7c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:47.549075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed722267-09fe-4189-8b65-026dbf1edea5 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:47.550122 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bd360-3139-4161-a4d2-d54cea3b7c13 None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:47.550409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bd360-3139-4161-a4d2-d54cea3b7c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:47.550686 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bd360-3139-4161-a4d2-d54cea3b7c13 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:47.551416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bd360-3139-4161-a4d2-d54cea3b7c13 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:47.553803 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bd360-3139-4161-a4d2-d54cea3b7c13 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:47.577582 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 17/34] 10.222.0.26 () {60 vars in 1131 bytes} [Thu Sep 22 14:43:47 2022] POST /identity/v3/roles => generated 225 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:43:47.578848 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bd360-3139-4161-a4d2-d54cea3b7c13 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:47.590821 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:47.612574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3c040-5c62-4c76-9c5b-a7a7c88e25fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:47.613582 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3c040-5c62-4c76-9c5b-a7a7c88e25fa None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:47.614109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3c040-5c62-4c76-9c5b-a7a7c88e25fa None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:47.614553 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b3c040-5c62-4c76-9c5b-a7a7c88e25fa None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:47.615723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3c040-5c62-4c76-9c5b-a7a7c88e25fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:47.616235 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3c040-5c62-4c76-9c5b-a7a7c88e25fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:47.620392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b3c040-5c62-4c76-9c5b-a7a7c88e25fa None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:47.644859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 18/35] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:47.654350 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3c040-5c62-4c76-9c5b-a7a7c88e25fa None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:47.703636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bd360-3139-4161-a4d2-d54cea3b7c13 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:47.746637 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399802b-e5ae-484f-bb1a-9509bdc58440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:47.747704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399802b-e5ae-484f-bb1a-9509bdc58440 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:47.748163 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399802b-e5ae-484f-bb1a-9509bdc58440 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:47.748616 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399802b-e5ae-484f-bb1a-9509bdc58440 None admin] PATH_INFO: `/v3/users/4c56fbc6ee754a96af39128dae8f43a4` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:47.750394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399802b-e5ae-484f-bb1a-9509bdc58440 None admin] RBAC: Authorizing `identity:get_user(user_id=4c56fbc6ee754a96af39128dae8f43a4)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:47.752074 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399802b-e5ae-484f-bb1a-9509bdc58440 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:47.754166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 18/36] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:43:47 2022] GET /identity/v3/users/4c56fbc6ee754a96af39128dae8f43a4 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:47.760070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399802b-e5ae-484f-bb1a-9509bdc58440 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:47.865432 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:47.889485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b3c040-5c62-4c76-9c5b-a7a7c88e25fa None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:47.897664 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399802b-e5ae-484f-bb1a-9509bdc58440 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:47.938894 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e8b5f-9f8a-4324-8f67-600e0410be8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:47.939856 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e8b5f-9f8a-4324-8f67-600e0410be8b None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:47.940895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e8b5f-9f8a-4324-8f67-600e0410be8b None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:47.941365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e8b5f-9f8a-4324-8f67-600e0410be8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:47.942859 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e8b5f-9f8a-4324-8f67-600e0410be8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:47.944626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e8b5f-9f8a-4324-8f67-600e0410be8b None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:47.945366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:47.947811 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 19/37] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:43:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:43:47.956105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e8b5f-9f8a-4324-8f67-600e0410be8b None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:47.960857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532648c-5120-4270-8da2-f0ee146d2a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:47.961444 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532648c-5120-4270-8da2-f0ee146d2a39 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:47.961920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532648c-5120-4270-8da2-f0ee146d2a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:47.962387 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532648c-5120-4270-8da2-f0ee146d2a39 None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:47.963570 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532648c-5120-4270-8da2-f0ee146d2a39 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:47.966233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532648c-5120-4270-8da2-f0ee146d2a39 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:48.081943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e8b5f-9f8a-4324-8f67-600e0410be8b None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:48.117496 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6148de-82d1-4eda-859a-cf3908bda4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:48.118111 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6148de-82d1-4eda-859a-cf3908bda4f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:48.118360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6148de-82d1-4eda-859a-cf3908bda4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:48.118614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6148de-82d1-4eda-859a-cf3908bda4f7 None admin] PATH_INFO: `/v3/domains/default/users/4c56fbc6ee754a96af39128dae8f43a4/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:48.120596 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6148de-82d1-4eda-859a-cf3908bda4f7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4c56fbc6ee754a96af39128dae8f43a4, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:48.122039 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6148de-82d1-4eda-859a-cf3908bda4f7 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:48.151082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 20/38] 10.222.0.26 () {58 vars in 1361 bytes} [Thu Sep 22 14:43:47 2022] PUT /identity/v3/domains/default/users/4c56fbc6ee754a96af39128dae8f43a4/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:43:48.151082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69834d-8405-4d7a-b157-e65ff302e99e None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:48.151082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69834d-8405-4d7a-b157-e65ff302e99e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:48.151082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69834d-8405-4d7a-b157-e65ff302e99e None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:48.151082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 21/39] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:48.151082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7cbab-cc90-41ad-8f0d-b80337a6faa3 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:48.151645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7cbab-cc90-41ad-8f0d-b80337a6faa3 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:48.151645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7cbab-cc90-41ad-8f0d-b80337a6faa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:48.154258 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:48.172501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-5db7cbab-cc90-41ad-8f0d-b80337a6faa3 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:48.174971 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-5db7cbab-cc90-41ad-8f0d-b80337a6faa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:48.246945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 19/40] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Sep 22 14:43:47 2022] POST /identity/v3/projects => generated 301 bytes in 594 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:43:48.259222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 22/41] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:43:48.319923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8599b-0fae-4191-bf10-774f4b2441b0 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:48.320447 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8599b-0fae-4191-bf10-774f4b2441b0 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:48.320956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8599b-0fae-4191-bf10-774f4b2441b0 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:48.324087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 20/42] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:48.331417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d863f7d-74a6-4b2e-98e1-5c40f58429d6 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:48.331417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d863f7d-74a6-4b2e-98e1-5c40f58429d6 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:48.331686 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d863f7d-74a6-4b2e-98e1-5c40f58429d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:48.358283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-3d863f7d-74a6-4b2e-98e1-5c40f58429d6 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:48.359605 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3d863f7d-74a6-4b2e-98e1-5c40f58429d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:48.447706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 23/43] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:43:48.539696 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8599b-0fae-4191-bf10-774f4b2441b0 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:48.719577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d863f7d-74a6-4b2e-98e1-5c40f58429d6 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:48.763388 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8599b-0fae-4191-bf10-774f4b2441b0 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:48.790268 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:48.801913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873823e-505e-484c-803a-16ff6df6eb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:48.802538 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873823e-505e-484c-803a-16ff6df6eb9c None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:48.802848 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873823e-505e-484c-803a-16ff6df6eb9c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:48.803123 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873823e-505e-484c-803a-16ff6df6eb9c None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:48.803808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873823e-505e-484c-803a-16ff6df6eb9c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:48.805327 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873823e-505e-484c-803a-16ff6df6eb9c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:48.822666 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 21/44] 10.222.0.26 () {60 vars in 1131 bytes} [Thu Sep 22 14:43:48 2022] POST /identity/v3/roles => generated 223 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:43:48.832937 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d863f7d-74a6-4b2e-98e1-5c40f58429d6 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:48.874933 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde013f-f327-4dde-bb4d-e8ec61ea5681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:48.875603 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde013f-f327-4dde-bb4d-e8ec61ea5681 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:48.875887 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde013f-f327-4dde-bb4d-e8ec61ea5681 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:48.876159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde013f-f327-4dde-bb4d-e8ec61ea5681 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:48.877465 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde013f-f327-4dde-bb4d-e8ec61ea5681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:48.879025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde013f-f327-4dde-bb4d-e8ec61ea5681 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:48.881046 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 24/45] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:43:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:43:48.886920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873823e-505e-484c-803a-16ff6df6eb9c None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:48.936018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:49.010561 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873823e-505e-484c-803a-16ff6df6eb9c None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:49.053175 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8b41d-1e58-41fb-998f-7f989e4e77c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:49.053823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8b41d-1e58-41fb-998f-7f989e4e77c7 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:49.054105 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8b41d-1e58-41fb-998f-7f989e4e77c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:49.054375 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8b41d-1e58-41fb-998f-7f989e4e77c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:49.055082 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8b41d-1e58-41fb-998f-7f989e4e77c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:49.056576 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8b41d-1e58-41fb-998f-7f989e4e77c7 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:49.090512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 22/46] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Sep 22 14:43:48 2022] POST /identity/v3/projects => generated 312 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:43:50.368304 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89c57d-f252-40d8-bf2a-ffda1d5730de None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:50.368809 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89c57d-f252-40d8-bf2a-ffda1d5730de None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:50.368976 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89c57d-f252-40d8-bf2a-ffda1d5730de None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:50.370320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 25/47] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:50.375756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afa925-1d1d-467c-ba24-c4c6cad38ad5 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:50.376188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afa925-1d1d-467c-ba24-c4c6cad38ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:50.376446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05afa925-1d1d-467c-ba24-c4c6cad38ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:50.396070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-05afa925-1d1d-467c-ba24-c4c6cad38ad5 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:50.397612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-05afa925-1d1d-467c-ba24-c4c6cad38ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:50.468026 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 23/48] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:50.736230 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89c57d-f252-40d8-bf2a-ffda1d5730de None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:50.839064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89c57d-f252-40d8-bf2a-ffda1d5730de None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:50.867902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:50.874457 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b2add-d9e7-439b-b293-359c7705ca3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:50.875127 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b2add-d9e7-439b-b293-359c7705ca3e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:50.875403 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b2add-d9e7-439b-b293-359c7705ca3e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:50.875671 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b2add-d9e7-439b-b293-359c7705ca3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:50.886908 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b2add-d9e7-439b-b293-359c7705ca3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:43:50.891843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-966b2add-d9e7-439b-b293-359c7705ca3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:43:50.891843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:50.891843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:50.891843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:50.891843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:50.891843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:50.891843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:50.891843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:50.891843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:50.891843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:50.891843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:50.891843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:50.891843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:43:50.891843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:43:50.891843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:43:50.891843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:43:50.892699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:43:50.892699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:43:50.892699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:43:50.892699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:43:50.892699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:50.892699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:50.892699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:50.892699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:50.892699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:50.892699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:50.892699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:50.892699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:50.892699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:50.892699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:50.892699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:50.892699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:50.893454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:50.893454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:50.893454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:50.893454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:50.893454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:43:50.893454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:43:50.893454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:43:50.893454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:43:50.893454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:43:50.893454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:43:50.893454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:43:50.893454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:43:50.893454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 26/49] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:43:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:43:50.898832 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05afa925-1d1d-467c-ba24-c4c6cad38ad5 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:50.922384 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:51.001277 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05afa925-1d1d-467c-ba24-c4c6cad38ad5 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:51.035256 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696beb0-116a-47de-94e7-6ba433600a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:51.035865 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696beb0-116a-47de-94e7-6ba433600a1e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:51.036144 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696beb0-116a-47de-94e7-6ba433600a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:51.036413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696beb0-116a-47de-94e7-6ba433600a1e None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:51.037081 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696beb0-116a-47de-94e7-6ba433600a1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:51.037382 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696beb0-116a-47de-94e7-6ba433600a1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:51.038786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696beb0-116a-47de-94e7-6ba433600a1e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:51.046947 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 24/50] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:43:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:51.053173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b2add-d9e7-439b-b293-359c7705ca3e None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:51.152394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b2add-d9e7-439b-b293-359c7705ca3e None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:51.188936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974b178-9d33-4e73-bbbb-a92ea24c2be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:51.189571 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974b178-9d33-4e73-bbbb-a92ea24c2be3 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:51.189880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974b178-9d33-4e73-bbbb-a92ea24c2be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:51.190159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974b178-9d33-4e73-bbbb-a92ea24c2be3 None admin] PATH_INFO: `/v3/users/4c56fbc6ee754a96af39128dae8f43a4` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:51.191332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974b178-9d33-4e73-bbbb-a92ea24c2be3 None admin] RBAC: Authorizing `identity:get_user(user_id=4c56fbc6ee754a96af39128dae8f43a4)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:51.192787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974b178-9d33-4e73-bbbb-a92ea24c2be3 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:51.194363 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 27/51] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:43:51 2022] GET /identity/v3/users/4c56fbc6ee754a96af39128dae8f43a4 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:51.199835 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696beb0-116a-47de-94e7-6ba433600a1e None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:51.283361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945519a-7d2c-4cbf-a35f-d901c943b17b None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:51.283566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945519a-7d2c-4cbf-a35f-d901c943b17b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:51.284309 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945519a-7d2c-4cbf-a35f-d901c943b17b None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:51.285643 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 28/52] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:43:51.288179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:51.290814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f2c6f-bb04-4e53-8391-c37aa2f9025a None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:51.291096 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f2c6f-bb04-4e53-8391-c37aa2f9025a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:51.291374 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f2c6f-bb04-4e53-8391-c37aa2f9025a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:51.309941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696beb0-116a-47de-94e7-6ba433600a1e None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:51.311875 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-ac9f2c6f-bb04-4e53-8391-c37aa2f9025a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:51.313131 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ac9f2c6f-bb04-4e53-8391-c37aa2f9025a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:51.346803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db7552-236d-493f-8214-5545ac56f220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:51.347447 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db7552-236d-493f-8214-5545ac56f220 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:51.347736 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db7552-236d-493f-8214-5545ac56f220 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:51.348012 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db7552-236d-493f-8214-5545ac56f220 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:51.349120 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db7552-236d-493f-8214-5545ac56f220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:51.350587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db7552-236d-493f-8214-5545ac56f220 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:51.352304 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 25/53] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:43:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:51.358846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db7552-236d-493f-8214-5545ac56f220 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:51.371569 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:51.385666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 29/54] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:51.460654 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db7552-236d-493f-8214-5545ac56f220 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:51.495239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89875b-f86b-497c-801f-2f5d21898021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:51.495839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89875b-f86b-497c-801f-2f5d21898021 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:51.496120 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89875b-f86b-497c-801f-2f5d21898021 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:51.496389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89875b-f86b-497c-801f-2f5d21898021 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:51.497112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89875b-f86b-497c-801f-2f5d21898021 None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.domain.id=default, user.id=4c56fbc6ee754a96af39128dae8f43a4) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:51.497418 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89875b-f86b-497c-801f-2f5d21898021 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:51.498902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89875b-f86b-497c-801f-2f5d21898021 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:51.507438 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 26/55] 10.222.0.26 () {56 vars in 1322 bytes} [Thu Sep 22 14:43:51 2022] GET /identity/v3/role_assignments?user.id=4c56fbc6ee754a96af39128dae8f43a4&scope.domain.id=default&role.id=32c8a81e339a462e973e76bb528ef35a => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:43:51.653890 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f2c6f-bb04-4e53-8391-c37aa2f9025a None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:51.762061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f2c6f-bb04-4e53-8391-c37aa2f9025a None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:51.800154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe593421-6d82-4077-baf8-86d646891934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:51.800798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe593421-6d82-4077-baf8-86d646891934 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:51.801081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe593421-6d82-4077-baf8-86d646891934 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:51.801358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe593421-6d82-4077-baf8-86d646891934 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:51.802517 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe593421-6d82-4077-baf8-86d646891934 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:51.804030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe593421-6d82-4077-baf8-86d646891934 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:51.805600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 30/56] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:43:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:51.811246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89875b-f86b-497c-801f-2f5d21898021 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:51.829838 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:51.910581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89875b-f86b-497c-801f-2f5d21898021 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:51.943617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce8353-3a23-4b90-8f23-0f84ddc73e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:51.944231 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce8353-3a23-4b90-8f23-0f84ddc73e2b None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:51.944515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce8353-3a23-4b90-8f23-0f84ddc73e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:51.944792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce8353-3a23-4b90-8f23-0f84ddc73e2b None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:51.945442 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce8353-3a23-4b90-8f23-0f84ddc73e2b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:51.946950 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce8353-3a23-4b90-8f23-0f84ddc73e2b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:51.973472 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 27/57] 10.222.0.26 () {60 vars in 1140 bytes} [Thu Sep 22 14:43:51 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:43:54.163835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc68b02-7128-41b4-8ec4-10f9ee9d6d3d None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:54.164352 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc68b02-7128-41b4-8ec4-10f9ee9d6d3d None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:54.164352 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc68b02-7128-41b4-8ec4-10f9ee9d6d3d None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:54.165699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 31/58] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:54.170803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21333bd5-738e-4798-bb0a-622a05c1f019 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:54.171172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21333bd5-738e-4798-bb0a-622a05c1f019 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:54.171355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21333bd5-738e-4798-bb0a-622a05c1f019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:54.189264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-21333bd5-738e-4798-bb0a-622a05c1f019 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:54.191016 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-21333bd5-738e-4798-bb0a-622a05c1f019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:54.251226 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:54.257817 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 28/59] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:54.531312 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc68b02-7128-41b4-8ec4-10f9ee9d6d3d None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:54.588756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:54.633283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc68b02-7128-41b4-8ec4-10f9ee9d6d3d None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:54.784622 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be568af5-816d-41f5-878c-dfed4d8d5607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:54.785457 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be568af5-816d-41f5-878c-dfed4d8d5607 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:54.785880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be568af5-816d-41f5-878c-dfed4d8d5607 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:54.786295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be568af5-816d-41f5-878c-dfed4d8d5607 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:54.787953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be568af5-816d-41f5-878c-dfed4d8d5607 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:54.790235 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be568af5-816d-41f5-878c-dfed4d8d5607 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:54.792716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 32/60] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:43:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:54.798312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21333bd5-738e-4798-bb0a-622a05c1f019 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:54.897829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21333bd5-738e-4798-bb0a-622a05c1f019 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:54.931485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7cee0-0947-4ba2-8ad3-0e779d0ffe95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:54.932102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7cee0-0947-4ba2-8ad3-0e779d0ffe95 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:54.932426 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7cee0-0947-4ba2-8ad3-0e779d0ffe95 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:54.932900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7cee0-0947-4ba2-8ad3-0e779d0ffe95 None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:54.933656 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc7cee0-0947-4ba2-8ad3-0e779d0ffe95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:54.935799 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc7cee0-0947-4ba2-8ad3-0e779d0ffe95 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:54.962595 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 29/61] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:43:54 2022] POST /identity/v3/users => generated 275 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:43:58.125372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d4b83-b26b-4bfb-9a1f-61155aa0f04e None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.125898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d4b83-b26b-4bfb-9a1f-61155aa0f04e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.125898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d4b83-b26b-4bfb-9a1f-61155aa0f04e None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.127453 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 33/62] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:58.134295 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6060595-5eb6-4257-bd8f-ec6d436514ca None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.135603 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6060595-5eb6-4257-bd8f-ec6d436514ca None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.136311 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6060595-5eb6-4257-bd8f-ec6d436514ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.175375 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-a6060595-5eb6-4257-bd8f-ec6d436514ca None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:58.176965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a6060595-5eb6-4257-bd8f-ec6d436514ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:58.254603 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:58.275262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 30/63] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:58.286313 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffa172-7262-4bf7-a869-26d2fbad637d None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.286924 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffa172-7262-4bf7-a869-26d2fbad637d None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.287418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffa172-7262-4bf7-a869-26d2fbad637d None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.288954 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 34/64] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:58.294360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b18612-97ee-421a-80a4-f2604918a865 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.294652 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b18612-97ee-421a-80a4-f2604918a865 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.294951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b18612-97ee-421a-80a4-f2604918a865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.336490 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-92b18612-97ee-421a-80a4-f2604918a865 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:58.337689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-92b18612-97ee-421a-80a4-f2604918a865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:58.436456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 31/65] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:58.555419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffa172-7262-4bf7-a869-26d2fbad637d None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:58.687945 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffa172-7262-4bf7-a869-26d2fbad637d None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:58.715707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:58.727788 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b176ef-0083-4578-94a1-434565720591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:58.728702 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b176ef-0083-4578-94a1-434565720591 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.728995 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b176ef-0083-4578-94a1-434565720591 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.729639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b176ef-0083-4578-94a1-434565720591 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.741939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b176ef-0083-4578-94a1-434565720591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:43:58.747146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-64b176ef-0083-4578-94a1-434565720591 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:43:58.747146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:58.747146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:58.747146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:58.747146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:58.747146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:58.747146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:58.747146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:58.747146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:58.747146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:58.747146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:58.747146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:58.747146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:43:58.747146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:43:58.747146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:43:58.747146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:43:58.748023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:43:58.748023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:43:58.748023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:43:58.748023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:43:58.748023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:58.748023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:58.748023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:58.748023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:58.748023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:58.748023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:58.748023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:58.748023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:58.748023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:58.748023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:58.748023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:58.748023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:58.748789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:58.748789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:58.748789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:58.748789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:58.748789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:43:58.748789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:43:58.748789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:43:58.748789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:43:58.748789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:43:58.748789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:43:58.748789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:43:58.748789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:43:58.748789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 35/66] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:43:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:43:58.754718 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b18612-97ee-421a-80a4-f2604918a865 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:58.822197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b176ef-0083-4578-94a1-434565720591 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:58.919064 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b18612-97ee-421a-80a4-f2604918a865 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:58.938635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:58.965556 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6080aa5-80db-4d0f-a58b-1bd699f4e076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:58.966521 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6080aa5-80db-4d0f-a58b-1bd699f4e076 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.966839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6080aa5-80db-4d0f-a58b-1bd699f4e076 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.967275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6080aa5-80db-4d0f-a58b-1bd699f4e076 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.967969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6080aa5-80db-4d0f-a58b-1bd699f4e076 None admin] RBAC: Adding query filter params (name=member) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:58.968282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6080aa5-80db-4d0f-a58b-1bd699f4e076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:58.969690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6080aa5-80db-4d0f-a58b-1bd699f4e076 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:58.973012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b176ef-0083-4578-94a1-434565720591 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:58.978868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 32/67] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:43:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:58.979941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b60519-0599-4c46-9108-cf37974b7495 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.980222 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b60519-0599-4c46-9108-cf37974b7495 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.980503 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b60519-0599-4c46-9108-cf37974b7495 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.981723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 33/68] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:58.988629 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa39f95-05ac-4abb-88b1-7506fc9786e5 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.988919 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa39f95-05ac-4abb-88b1-7506fc9786e5 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.989197 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa39f95-05ac-4abb-88b1-7506fc9786e5 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:58.990524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 34/69] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:58.991900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad23c7f-7895-4147-ab4b-79e3fbb6f6d4 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:58.991900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad23c7f-7895-4147-ab4b-79e3fbb6f6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:58.992168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad23c7f-7895-4147-ab4b-79e3fbb6f6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.010482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-1ad23c7f-7895-4147-ab4b-79e3fbb6f6d4 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:59.011727 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1ad23c7f-7895-4147-ab4b-79e3fbb6f6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:59.012248 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857da2a-e9e1-444a-a2ef-f815b0165a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.012890 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857da2a-e9e1-444a-a2ef-f815b0165a27 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.013173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857da2a-e9e1-444a-a2ef-f815b0165a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.013436 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857da2a-e9e1-444a-a2ef-f815b0165a27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.024778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857da2a-e9e1-444a-a2ef-f815b0165a27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:43:59.030289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-f857da2a-e9e1-444a-a2ef-f815b0165a27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:43:59.030289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:59.030289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:59.030289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:59.030289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:59.030289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:59.030289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:59.030289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:59.030289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:59.030289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:59.030289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:59.030289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:59.030289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:43:59.030289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:43:59.030289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:43:59.030289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:43:59.035108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:43:59.035108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:43:59.035108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:43:59.035108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:43:59.035108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:59.035108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:59.035108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:59.035108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:59.035108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:59.035108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:59.035108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:59.035108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:59.035108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:59.035108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:59.035108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:59.035108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:59.036005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:59.036005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:59.036005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:59.036005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:59.036005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:43:59.036005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:43:59.036005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:43:59.036005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:43:59.036005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:43:59.036005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:43:59.036005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:43:59.036005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:43:59.036005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 36/70] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:43:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:43:59.036005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857da2a-e9e1-444a-a2ef-f815b0165a27 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.088376 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:59.103046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 35/71] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:59.105554 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eef4bd-a6e8-4855-b58c-7db2a1018cc2 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.105847 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eef4bd-a6e8-4855-b58c-7db2a1018cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.106129 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eef4bd-a6e8-4855-b58c-7db2a1018cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.131494 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-00eef4bd-a6e8-4855-b58c-7db2a1018cc2 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:59.133007 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-00eef4bd-a6e8-4855-b58c-7db2a1018cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:59.150906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857da2a-e9e1-444a-a2ef-f815b0165a27 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:59.206462 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54494664-d2e8-4518-8fe2-c93799d11b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.207057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54494664-d2e8-4518-8fe2-c93799d11b2e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.207350 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54494664-d2e8-4518-8fe2-c93799d11b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.207624 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54494664-d2e8-4518-8fe2-c93799d11b2e None admin] PATH_INFO: `/v3/users/b0b665d93f28427a85c1b52614174fb7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.218633 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 36/72] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:59.224578 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54494664-d2e8-4518-8fe2-c93799d11b2e None admin] RBAC: Authorizing `identity:get_user(user_id=b0b665d93f28427a85c1b52614174fb7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:59.226049 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54494664-d2e8-4518-8fe2-c93799d11b2e None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:59.227703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eef4bd-a6e8-4855-b58c-7db2a1018cc2 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.227810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 37/73] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:43:59 2022] GET /identity/v3/users/b0b665d93f28427a85c1b52614174fb7 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:59.229168 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62d91f-bb03-4751-ac2c-c2680a5e9fec None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.229453 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62d91f-bb03-4751-ac2c-c2680a5e9fec None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.229729 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62d91f-bb03-4751-ac2c-c2680a5e9fec None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.231100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 38/74] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:43:59.235960 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62d91f-bb03-4751-ac2c-c2680a5e9fec None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.276110 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:59.360018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eef4bd-a6e8-4855-b58c-7db2a1018cc2 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:59.369590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62d91f-bb03-4751-ac2c-c2680a5e9fec None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:59.371905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:59.397184 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06731c26-22ed-4569-8d4b-1cffafa8dfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.397822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06731c26-22ed-4569-8d4b-1cffafa8dfbb None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.398072 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06731c26-22ed-4569-8d4b-1cffafa8dfbb None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.398343 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06731c26-22ed-4569-8d4b-1cffafa8dfbb None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.399037 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06731c26-22ed-4569-8d4b-1cffafa8dfbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:59.399337 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06731c26-22ed-4569-8d4b-1cffafa8dfbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:59.400699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06731c26-22ed-4569-8d4b-1cffafa8dfbb None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:59.409865 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 37/75] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:43:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:59.411109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c05aac-d022-48d3-bdd5-ab12af5a1d4c None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.411374 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c05aac-d022-48d3-bdd5-ab12af5a1d4c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.411649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c05aac-d022-48d3-bdd5-ab12af5a1d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.412728 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20975f3-be8b-4967-9544-2e732a0ad104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.413339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20975f3-be8b-4967-9544-2e732a0ad104 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.413619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20975f3-be8b-4967-9544-2e732a0ad104 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.413892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20975f3-be8b-4967-9544-2e732a0ad104 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.433078 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-c0c05aac-d022-48d3-bdd5-ab12af5a1d4c None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:43:59.434825 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c0c05aac-d022-48d3-bdd5-ab12af5a1d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:43:59.486768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20975f3-be8b-4967-9544-2e732a0ad104 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:59.488723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20975f3-be8b-4967-9544-2e732a0ad104 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:59.506269 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 38/76] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:43:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:43:59.507148 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c05aac-d022-48d3-bdd5-ab12af5a1d4c None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.546369 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:59.599158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 39/77] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:43:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:43:59.600327 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20975f3-be8b-4967-9544-2e732a0ad104 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.603257 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:59.627996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c05aac-d022-48d3-bdd5-ab12af5a1d4c None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:59.666530 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a8bb6-981b-4e58-93b7-ca010ad2567f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.667201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a8bb6-981b-4e58-93b7-ca010ad2567f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.667487 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a8bb6-981b-4e58-93b7-ca010ad2567f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.667757 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a8bb6-981b-4e58-93b7-ca010ad2567f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.679823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a8bb6-981b-4e58-93b7-ca010ad2567f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:43:59.686590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-b38a8bb6-981b-4e58-93b7-ca010ad2567f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:43:59.686590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:59.686590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:59.686590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:59.686590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:59.686590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:59.686590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:59.686590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:59.686590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:59.686590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:59.686590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:59.686590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:59.686590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:43:59.686590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:43:59.686590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:43:59.686590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:43:59.687492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:43:59.687492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:43:59.687492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:43:59.687492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:43:59.687492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:59.687492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:59.687492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:59.687492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:59.687492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:59.687492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:59.687492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:59.687492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:59.687492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:59.687492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:59.687492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:59.687492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:59.688262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:59.688262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:59.688262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:59.688262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:59.688262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:43:59.688262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:43:59.688262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:43:59.688262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:43:59.688262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:43:59.688262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:43:59.688262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:43:59.688262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:43:59.688262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 39/78] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:43:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:43:59.688893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a8bb6-981b-4e58-93b7-ca010ad2567f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.712266 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20975f3-be8b-4967-9544-2e732a0ad104 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:59.748668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090bebb-2487-4ebc-b12a-8870e205c146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.749234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090bebb-2487-4ebc-b12a-8870e205c146 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.749515 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090bebb-2487-4ebc-b12a-8870e205c146 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.749792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090bebb-2487-4ebc-b12a-8870e205c146 None admin] PATH_INFO: `/v3/users/4c56fbc6ee754a96af39128dae8f43a4` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.750964 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090bebb-2487-4ebc-b12a-8870e205c146 None admin] RBAC: Authorizing `identity:get_user(user_id=4c56fbc6ee754a96af39128dae8f43a4)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:59.752445 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090bebb-2487-4ebc-b12a-8870e205c146 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:59.754193 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 40/79] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:43:59 2022] GET /identity/v3/users/4c56fbc6ee754a96af39128dae8f43a4 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:59.755964 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090bebb-2487-4ebc-b12a-8870e205c146 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.782271 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:59.795193 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a8bb6-981b-4e58-93b7-ca010ad2567f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:59.829620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4caf83c-77c7-4c30-8e3d-9b679cdeada1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.830252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4caf83c-77c7-4c30-8e3d-9b679cdeada1 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.830613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4caf83c-77c7-4c30-8e3d-9b679cdeada1 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.830905 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4caf83c-77c7-4c30-8e3d-9b679cdeada1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.831768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:43:59.843264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4caf83c-77c7-4c30-8e3d-9b679cdeada1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:43:59.852961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-c4caf83c-77c7-4c30-8e3d-9b679cdeada1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:43:59.852961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:43:59.852961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:43:59.852961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:43:59.852961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:43:59.852961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:43:59.852961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:43:59.852961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:43:59.852961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:43:59.852961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:43:59.852961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:43:59.852961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:43:59.852961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:43:59.852961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:43:59.852961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:43:59.852961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:43:59.853893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:43:59.853893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:43:59.853893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:43:59.853893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:43:59.853893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:43:59.853893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:43:59.853893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:43:59.853893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:43:59.853893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:43:59.853893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:43:59.853893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:43:59.853893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:43:59.853893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:43:59.853893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:43:59.853893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:43:59.853893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:43:59.855630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:43:59.855630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:43:59.855630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:43:59.855630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:43:59.855630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:43:59.855630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:43:59.855630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:43:59.855630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:43:59.855630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:43:59.855630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:43:59.855630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:43:59.855630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:43:59.855630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 40/80] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:43:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:43:59.855630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4caf83c-77c7-4c30-8e3d-9b679cdeada1 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.869119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090bebb-2487-4ebc-b12a-8870e205c146 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:59.904815 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5802c7-529c-4c3a-ba51-5067f85101bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.905454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5802c7-529c-4c3a-ba51-5067f85101bd None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.905732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5802c7-529c-4c3a-ba51-5067f85101bd None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.906001 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5802c7-529c-4c3a-ba51-5067f85101bd None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.907498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5802c7-529c-4c3a-ba51-5067f85101bd None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:43:59.909179 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5802c7-529c-4c3a-ba51-5067f85101bd None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:43:59.910867 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 41/81] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:43:59 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:43:59.911644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5802c7-529c-4c3a-ba51-5067f85101bd None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:43:59.961696 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4caf83c-77c7-4c30-8e3d-9b679cdeada1 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:43:59.996519 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fb11f-ad32-4f95-9ed9-af08f420fe7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:43:59.997134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fb11f-ad32-4f95-9ed9-af08f420fe7e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:43:59.997414 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fb11f-ad32-4f95-9ed9-af08f420fe7e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:43:59.997690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fb11f-ad32-4f95-9ed9-af08f420fe7e None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:43:59.998359 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fb11f-ad32-4f95-9ed9-af08f420fe7e None admin] RBAC: Adding query filter params (name=member) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:43:59.998667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fb11f-ad32-4f95-9ed9-af08f420fe7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.000146 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fb11f-ad32-4f95-9ed9-af08f420fe7e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.008474 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 41/82] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:43:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:00.009292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fb11f-ad32-4f95-9ed9-af08f420fe7e None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:00.013877 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5802c7-529c-4c3a-ba51-5067f85101bd None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:00.023396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:00.049103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4dcf51-b934-46b9-9a9a-34439c5df253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:00.050267 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4dcf51-b934-46b9-9a9a-34439c5df253 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:00.050551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4dcf51-b934-46b9-9a9a-34439c5df253 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:00.050865 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4dcf51-b934-46b9-9a9a-34439c5df253 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:00.059012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:00.116463 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fb11f-ad32-4f95-9ed9-af08f420fe7e None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:00.120514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4dcf51-b934-46b9-9a9a-34439c5df253 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.122015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4dcf51-b934-46b9-9a9a-34439c5df253 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.151688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db5a1d-df6f-4888-9caf-a10257503d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:00.152335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db5a1d-df6f-4888-9caf-a10257503d65 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:00.152616 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db5a1d-df6f-4888-9caf-a10257503d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:00.152927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db5a1d-df6f-4888-9caf-a10257503d65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:00.154043 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db5a1d-df6f-4888-9caf-a10257503d65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.155546 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db5a1d-df6f-4888-9caf-a10257503d65 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.157145 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 42/83] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:00.157910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db5a1d-df6f-4888-9caf-a10257503d65 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:00.238648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 42/84] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:43:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:00.240209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4dcf51-b934-46b9-9a9a-34439c5df253 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:00.242225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:00.266176 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db5a1d-df6f-4888-9caf-a10257503d65 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:00.285282 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:00.302183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a10658-67cf-4dd0-bf02-6b3816b28526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:00.302838 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a10658-67cf-4dd0-bf02-6b3816b28526 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:00.303123 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a10658-67cf-4dd0-bf02-6b3816b28526 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:00.303399 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a10658-67cf-4dd0-bf02-6b3816b28526 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:00.304070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a10658-67cf-4dd0-bf02-6b3816b28526 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:00.304376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a10658-67cf-4dd0-bf02-6b3816b28526 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.305786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a10658-67cf-4dd0-bf02-6b3816b28526 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.314974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 43/85] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:00.316156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a10658-67cf-4dd0-bf02-6b3816b28526 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:00.353570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4dcf51-b934-46b9-9a9a-34439c5df253 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:00.389322 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f509f-7176-4510-bed1-bb9fdbeed4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:00.389921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f509f-7176-4510-bed1-bb9fdbeed4cf None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:00.390200 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f509f-7176-4510-bed1-bb9fdbeed4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:00.390470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f509f-7176-4510-bed1-bb9fdbeed4cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:00.391228 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f509f-7176-4510-bed1-bb9fdbeed4cf None admin] RBAC: Adding query filter params (role.id=7d3cf58c1ad2407c9f8c60ba99b2006d, scope.project.id=886ef67601844188bb0aa19dcf32dea7, user.id=b0b665d93f28427a85c1b52614174fb7) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:00.391536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f509f-7176-4510-bed1-bb9fdbeed4cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.393049 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f509f-7176-4510-bed1-bb9fdbeed4cf None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.398416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 43/86] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:00 2022] GET /identity/v3/role_assignments?user.id=b0b665d93f28427a85c1b52614174fb7&scope.project.id=886ef67601844188bb0aa19dcf32dea7&role.id=7d3cf58c1ad2407c9f8c60ba99b2006d => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:00.399385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f509f-7176-4510-bed1-bb9fdbeed4cf None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:00.419868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a10658-67cf-4dd0-bf02-6b3816b28526 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:00.460669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325216c-3098-4571-a77b-22733ba57b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:00.461364 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325216c-3098-4571-a77b-22733ba57b1e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:00.461648 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325216c-3098-4571-a77b-22733ba57b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:00.461925 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325216c-3098-4571-a77b-22733ba57b1e None admin] PATH_INFO: `/v3/users/b0b665d93f28427a85c1b52614174fb7` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:00.463262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325216c-3098-4571-a77b-22733ba57b1e None admin] RBAC: Authorizing `identity:get_user(user_id=b0b665d93f28427a85c1b52614174fb7)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.464782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325216c-3098-4571-a77b-22733ba57b1e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.466481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 44/87] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:00 2022] GET /identity/v3/users/b0b665d93f28427a85c1b52614174fb7 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:00.467319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325216c-3098-4571-a77b-22733ba57b1e None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:00.480915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:00.537418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f509f-7176-4510-bed1-bb9fdbeed4cf None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:00.539511 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:00.578432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325216c-3098-4571-a77b-22733ba57b1e None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:00.581936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073aa2e-6923-412c-a6ff-087b66c37eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:00.582726 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073aa2e-6923-412c-a6ff-087b66c37eb9 None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:00.583065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073aa2e-6923-412c-a6ff-087b66c37eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:00.583339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073aa2e-6923-412c-a6ff-087b66c37eb9 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:00.584024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073aa2e-6923-412c-a6ff-087b66c37eb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.585469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073aa2e-6923-412c-a6ff-087b66c37eb9 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.618923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 44/88] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:00 2022] POST /identity/v3/users => generated 289 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:00.619930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073aa2e-6923-412c-a6ff-087b66c37eb9 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:00.626934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f4e89-ca1c-49e1-8e93-803f7a879d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:00.628825 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f4e89-ca1c-49e1-8e93-803f7a879d28 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:00.629106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f4e89-ca1c-49e1-8e93-803f7a879d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:00.629380 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f4e89-ca1c-49e1-8e93-803f7a879d28 None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:00.635488 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f4e89-ca1c-49e1-8e93-803f7a879d28 None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.637007 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f4e89-ca1c-49e1-8e93-803f7a879d28 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.644742 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 45/89] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:00 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:00.644742 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f4e89-ca1c-49e1-8e93-803f7a879d28 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:00.753231 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:00.776561 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073aa2e-6923-412c-a6ff-087b66c37eb9 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:00.792419 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f4e89-ca1c-49e1-8e93-803f7a879d28 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:00.817892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403b3bc-ee91-4a7d-8792-3c7e5f719d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:00.818544 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403b3bc-ee91-4a7d-8792-3c7e5f719d22 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:00.818867 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403b3bc-ee91-4a7d-8792-3c7e5f719d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:00.819146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403b3bc-ee91-4a7d-8792-3c7e5f719d22 None admin] PATH_INFO: `/v3/users/b0b665d93f28427a85c1b52614174fb7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:00.820334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403b3bc-ee91-4a7d-8792-3c7e5f719d22 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b665d93f28427a85c1b52614174fb7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.821963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403b3bc-ee91-4a7d-8792-3c7e5f719d22 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.823681 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 45/90] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:00 2022] GET /identity/v3/users/b0b665d93f28427a85c1b52614174fb7 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:00.824464 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403b3bc-ee91-4a7d-8792-3c7e5f719d22 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:00.839493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e64cd3-ad7c-4d04-a34f-1f345f67fb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:00.840149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e64cd3-ad7c-4d04-a34f-1f345f67fb70 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:00.840438 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e64cd3-ad7c-4d04-a34f-1f345f67fb70 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:00.840715 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e64cd3-ad7c-4d04-a34f-1f345f67fb70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:00.857820 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:00.927623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e64cd3-ad7c-4d04-a34f-1f345f67fb70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:00.929108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e64cd3-ad7c-4d04-a34f-1f345f67fb70 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:00.952572 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403b3bc-ee91-4a7d-8792-3c7e5f719d22 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:00.996277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21a775-c421-46b0-9b11-205a9190fa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:00.997449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21a775-c421-46b0-9b11-205a9190fa8c None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:00.998367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21a775-c421-46b0-9b11-205a9190fa8c None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:00.998908 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21a775-c421-46b0-9b11-205a9190fa8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:01.000284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21a775-c421-46b0-9b11-205a9190fa8c None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=886ef67601844188bb0aa19dcf32dea7, user.id=4c56fbc6ee754a96af39128dae8f43a4) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:01.000284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21a775-c421-46b0-9b11-205a9190fa8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:01.002766 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21a775-c421-46b0-9b11-205a9190fa8c None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:01.008856 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 46/91] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:00 2022] GET /identity/v3/role_assignments?user.id=4c56fbc6ee754a96af39128dae8f43a4&scope.project.id=886ef67601844188bb0aa19dcf32dea7&role.id=32c8a81e339a462e973e76bb528ef35a => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:01.010411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21a775-c421-46b0-9b11-205a9190fa8c None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:01.043320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:01.073914 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 46/92] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:01.079837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e64cd3-ad7c-4d04-a34f-1f345f67fb70 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:01.135789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:01.149783 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21a775-c421-46b0-9b11-205a9190fa8c None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:01.198596 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14b2cb-cd10-4e0c-b469-8f5b96b755c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:01.200053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14b2cb-cd10-4e0c-b469-8f5b96b755c4 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:01.200490 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14b2cb-cd10-4e0c-b469-8f5b96b755c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:01.201095 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14b2cb-cd10-4e0c-b469-8f5b96b755c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:01.207212 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e64cd3-ad7c-4d04-a34f-1f345f67fb70 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:01.269436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea304df8-2a5b-4d81-8d41-a91d41c79224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:01.270186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea304df8-2a5b-4d81-8d41-a91d41c79224 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:01.270353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea304df8-2a5b-4d81-8d41-a91d41c79224 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:01.270630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea304df8-2a5b-4d81-8d41-a91d41c79224 None admin] PATH_INFO: `/v3/projects/1da4a7c776cc4b72afc891ddabce4d99` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:01.276229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea304df8-2a5b-4d81-8d41-a91d41c79224 None admin] RBAC: Authorizing `identity:get_project(project_id=1da4a7c776cc4b72afc891ddabce4d99)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:01.277702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea304df8-2a5b-4d81-8d41-a91d41c79224 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:01.280303 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 47/93] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:01 2022] GET /identity/v3/projects/1da4a7c776cc4b72afc891ddabce4d99 => generated 312 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:01.286981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea304df8-2a5b-4d81-8d41-a91d41c79224 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:01.324295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14b2cb-cd10-4e0c-b469-8f5b96b755c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:01.325805 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14b2cb-cd10-4e0c-b469-8f5b96b755c4 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:01.380796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:01.490430 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea304df8-2a5b-4d81-8d41-a91d41c79224 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:01.506640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:01.517864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 47/94] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:01.524550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14b2cb-cd10-4e0c-b469-8f5b96b755c4 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:01.546513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6eb0a4-379d-471e-adf9-2d01b555ea30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:01.547545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6eb0a4-379d-471e-adf9-2d01b555ea30 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:01.548051 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6eb0a4-379d-471e-adf9-2d01b555ea30 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:01.548522 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6eb0a4-379d-471e-adf9-2d01b555ea30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:01.550015 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6eb0a4-379d-471e-adf9-2d01b555ea30 None admin] RBAC: Adding query filter params (role.id=7d3cf58c1ad2407c9f8c60ba99b2006d, scope.project.id=1da4a7c776cc4b72afc891ddabce4d99, user.id=b0b665d93f28427a85c1b52614174fb7) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:01.550911 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6eb0a4-379d-471e-adf9-2d01b555ea30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:01.552584 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6eb0a4-379d-471e-adf9-2d01b555ea30 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:01.559280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 48/95] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:01 2022] GET /identity/v3/role_assignments?user.id=b0b665d93f28427a85c1b52614174fb7&scope.project.id=1da4a7c776cc4b72afc891ddabce4d99&role.id=7d3cf58c1ad2407c9f8c60ba99b2006d => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:01.692099 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14b2cb-cd10-4e0c-b469-8f5b96b755c4 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:01.731217 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57db34f-0ef7-4a7d-936f-8a563d8d9d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:01.731865 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57db34f-0ef7-4a7d-936f-8a563d8d9d9a None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:01.732161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57db34f-0ef7-4a7d-936f-8a563d8d9d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:01.732425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57db34f-0ef7-4a7d-936f-8a563d8d9d9a None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:01.733588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57db34f-0ef7-4a7d-936f-8a563d8d9d9a None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:01.735110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57db34f-0ef7-4a7d-936f-8a563d8d9d9a None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:01.737655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 48/96] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:01 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:01.743966 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6eb0a4-379d-471e-adf9-2d01b555ea30 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:01.869793 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6eb0a4-379d-471e-adf9-2d01b555ea30 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:01.879980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:01.907255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad1283-693e-4f8e-a58e-d055c808a758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:01.907874 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad1283-693e-4f8e-a58e-d055c808a758 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:01.908156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad1283-693e-4f8e-a58e-d055c808a758 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:01.908430 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad1283-693e-4f8e-a58e-d055c808a758 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:01.909199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad1283-693e-4f8e-a58e-d055c808a758 None admin] RBAC: Adding query filter params (role.id=1b632105fdba460b809a2ecd1d186409, scope.project.id=886ef67601844188bb0aa19dcf32dea7, user.id=b0b665d93f28427a85c1b52614174fb7) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:01.909509 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad1283-693e-4f8e-a58e-d055c808a758 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:01.910981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad1283-693e-4f8e-a58e-d055c808a758 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:01.917196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 49/97] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:01 2022] GET /identity/v3/role_assignments?user.id=b0b665d93f28427a85c1b52614174fb7&scope.project.id=886ef67601844188bb0aa19dcf32dea7&role.id=1b632105fdba460b809a2ecd1d186409 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:03.676534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87da0c-2c38-49cf-9d0d-da9640220bf0 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:03.677023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87da0c-2c38-49cf-9d0d-da9640220bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:03.677023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87da0c-2c38-49cf-9d0d-da9640220bf0 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:03.678397 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 49/98] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:03.683927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66693cc-dd4d-4dee-ad4e-89e7e8d93900 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:03.684342 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66693cc-dd4d-4dee-ad4e-89e7e8d93900 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:03.684512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66693cc-dd4d-4dee-ad4e-89e7e8d93900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:03.711797 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-f66693cc-dd4d-4dee-ad4e-89e7e8d93900 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:03.713013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f66693cc-dd4d-4dee-ad4e-89e7e8d93900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:03.753164 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb2ab3-66e5-439a-958c-f69687de2991 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:03.753780 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb2ab3-66e5-439a-958c-f69687de2991 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:03.754255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb2ab3-66e5-439a-958c-f69687de2991 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:03.755853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 50/99] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:03.761550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86829c-d239-4d96-a708-f9d40d4d5567 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:03.762117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86829c-d239-4d96-a708-f9d40d4d5567 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:03.762591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86829c-d239-4d96-a708-f9d40d4d5567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:03.783230 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-2e86829c-d239-4d96-a708-f9d40d4d5567 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:03.784832 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-2e86829c-d239-4d96-a708-f9d40d4d5567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:03.826759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 50/100] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:03.867354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:03.890884 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 51/101] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:03.988940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b951fe6-7bca-4bd1-b0c1-f6886a69802c None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:03.989548 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b951fe6-7bca-4bd1-b0c1-f6886a69802c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:03.990021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b951fe6-7bca-4bd1-b0c1-f6886a69802c None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:03.991628 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 51/102] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:03.997311 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07398338-e7b5-40dc-91ed-b2992d937f88 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:03.997598 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07398338-e7b5-40dc-91ed-b2992d937f88 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:03.997927 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07398338-e7b5-40dc-91ed-b2992d937f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.022173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-07398338-e7b5-40dc-91ed-b2992d937f88 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:04.023413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-07398338-e7b5-40dc-91ed-b2992d937f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:04.092802 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 52/103] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:04.217869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b951fe6-7bca-4bd1-b0c1-f6886a69802c None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:04.268914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07398338-e7b5-40dc-91ed-b2992d937f88 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:04.383694 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b951fe6-7bca-4bd1-b0c1-f6886a69802c None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:04.385795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:04.454381 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a79e21-aa3d-44bf-b196-a61417a99e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:04.455064 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a79e21-aa3d-44bf-b196-a61417a99e73 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.455784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a79e21-aa3d-44bf-b196-a61417a99e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.456282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a79e21-aa3d-44bf-b196-a61417a99e73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.482560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a79e21-aa3d-44bf-b196-a61417a99e73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:04.489649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-69a79e21-aa3d-44bf-b196-a61417a99e73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:04.489649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:04.489649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:04.489649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:04.489649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:04.489649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:04.489649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:04.489649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:04.489649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:04.489649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:04.489649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:04.489649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:04.489649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:04.489649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:04.489649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:04.489649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:04.490523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:04.490523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:04.490523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:04.490523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:04.490523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:04.490523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:04.490523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:04.490523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:04.490523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:04.490523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:04.490523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:04.490523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:04.490523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:04.490523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:04.490523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:04.490523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:04.491329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:04.491329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:04.491329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:04.491329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:04.491329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:04.491329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:04.491329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:04.491329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:04.491329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:04.491329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:04.491329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:04.491329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:04.493894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 52/104] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:44:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:04.496808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a79e21-aa3d-44bf-b196-a61417a99e73 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:04.523973 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07398338-e7b5-40dc-91ed-b2992d937f88 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:04.545926 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:04.566751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860eba8-51a8-4813-8e45-f20857409bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:04.567395 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860eba8-51a8-4813-8e45-f20857409bc9 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.567682 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860eba8-51a8-4813-8e45-f20857409bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.567956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860eba8-51a8-4813-8e45-f20857409bc9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.581528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860eba8-51a8-4813-8e45-f20857409bc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:04.586761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-8860eba8-51a8-4813-8e45-f20857409bc9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:04.586761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:04.586761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:04.586761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:04.586761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:04.586761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:04.586761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:04.586761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:04.586761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:04.586761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:04.586761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:04.586761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:04.586761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:04.586761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:04.586761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:04.586761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:04.587655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:04.587655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:04.587655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:04.587655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:04.587655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:04.587655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:04.587655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:04.587655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:04.587655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:04.587655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:04.587655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:04.587655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:04.587655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:04.587655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:04.587655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:04.587655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:04.588433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:04.588433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:04.588433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:04.588433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:04.588433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:04.588433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:04.588433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:04.588433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:04.588433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:04.588433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:04.588433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:04.588433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:04.588433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 53/105] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:04.589846 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc27bb-c65e-47fb-97b0-5a45b4bf99af None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.590139 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc27bb-c65e-47fb-97b0-5a45b4bf99af None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.590417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc27bb-c65e-47fb-97b0-5a45b4bf99af None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.591702 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 54/106] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:04.596992 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc27bb-c65e-47fb-97b0-5a45b4bf99af None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:04.638325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a79e21-aa3d-44bf-b196-a61417a99e73 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:04.705688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27dd71-8064-42b6-ae8e-815e84a5433d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:04.708070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27dd71-8064-42b6-ae8e-815e84a5433d None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.708862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27dd71-8064-42b6-ae8e-815e84a5433d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.709553 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27dd71-8064-42b6-ae8e-815e84a5433d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.712124 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc27bb-c65e-47fb-97b0-5a45b4bf99af None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:04.732336 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27dd71-8064-42b6-ae8e-815e84a5433d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:04.743529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-fe27dd71-8064-42b6-ae8e-815e84a5433d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:04.743529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:04.743529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:04.743529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:04.743529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:04.743529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:04.743529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:04.743529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:04.743529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:04.743529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:04.743529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:04.743529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:04.743529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:04.743529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:04.743529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:04.743529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:04.745073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:04.745073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:04.745073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:04.745073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:04.745073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:04.745073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:04.745073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:04.745073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:04.745073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:04.745073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:04.745073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:04.745073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:04.745073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:04.745073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:04.745073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:04.745073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:04.746544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:04.746544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:04.746544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:04.746544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:04.746544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:04.746544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:04.746544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:04.746544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:04.746544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:04.746544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:04.746544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:04.746544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:04.751054 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 53/107] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:04.752245 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869caf83-3c3b-41ca-89bc-411a2c8d3bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:04.753365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869caf83-3c3b-41ca-89bc-411a2c8d3bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.753648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869caf83-3c3b-41ca-89bc-411a2c8d3bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.753920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869caf83-3c3b-41ca-89bc-411a2c8d3bd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.754489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27dd71-8064-42b6-ae8e-815e84a5433d None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:04.754633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869caf83-3c3b-41ca-89bc-411a2c8d3bd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:04.754947 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869caf83-3c3b-41ca-89bc-411a2c8d3bd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:04.756710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869caf83-3c3b-41ca-89bc-411a2c8d3bd8 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:04.769443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 55/108] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:04.775505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ac140-f9d2-49bd-bfde-c07ee917dee0 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.775710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ac140-f9d2-49bd-bfde-c07ee917dee0 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.776038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ac140-f9d2-49bd-bfde-c07ee917dee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.796287 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-5f3ac140-f9d2-49bd-bfde-c07ee917dee0 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:04.797529 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-5f3ac140-f9d2-49bd-bfde-c07ee917dee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:04.806230 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:04.838948 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:04.888812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 56/109] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:04.888812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ac140-f9d2-49bd-bfde-c07ee917dee0 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:04.929166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27dd71-8064-42b6-ae8e-815e84a5433d None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:04.994721 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f45f8-f483-43d9-b5b0-b4df19c704b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:04.995472 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f45f8-f483-43d9-b5b0-b4df19c704b3 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:04.995855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f45f8-f483-43d9-b5b0-b4df19c704b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:04.996142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f45f8-f483-43d9-b5b0-b4df19c704b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:04.996819 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f45f8-f483-43d9-b5b0-b4df19c704b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:04.997127 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f45f8-f483-43d9-b5b0-b4df19c704b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:04.998724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f45f8-f483-43d9-b5b0-b4df19c704b3 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.002079 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ac140-f9d2-49bd-bfde-c07ee917dee0 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.012743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 54/110] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:05.013661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f45f8-f483-43d9-b5b0-b4df19c704b3 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.043008 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533a8ba-3b18-43d1-af0a-be13e8236dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.043661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533a8ba-3b18-43d1-af0a-be13e8236dbc None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.043945 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533a8ba-3b18-43d1-af0a-be13e8236dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.044333 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533a8ba-3b18-43d1-af0a-be13e8236dbc None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.045011 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533a8ba-3b18-43d1-af0a-be13e8236dbc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:05.045314 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533a8ba-3b18-43d1-af0a-be13e8236dbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.046993 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c533a8ba-3b18-43d1-af0a-be13e8236dbc None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.057418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 57/111] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:05.058843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c349e6-de22-46d9-b5cb-02129317c36b None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.059137 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c349e6-de22-46d9-b5cb-02129317c36b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.059433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c349e6-de22-46d9-b5cb-02129317c36b None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.060815 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 58/112] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:05.061571 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c349e6-de22-46d9-b5cb-02129317c36b None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.099660 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:05.110759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:05.133637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f45f8-f483-43d9-b5b0-b4df19c704b3 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.181492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10299c-c9da-4fc4-8837-683351eed6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.182139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10299c-c9da-4fc4-8837-683351eed6bc None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.182427 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10299c-c9da-4fc4-8837-683351eed6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.182705 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10299c-c9da-4fc4-8837-683351eed6bc None admin] PATH_INFO: `/v3/users/4c56fbc6ee754a96af39128dae8f43a4` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.183904 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c349e6-de22-46d9-b5cb-02129317c36b None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.184021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10299c-c9da-4fc4-8837-683351eed6bc None admin] RBAC: Authorizing `identity:get_user(user_id=4c56fbc6ee754a96af39128dae8f43a4)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.185550 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10299c-c9da-4fc4-8837-683351eed6bc None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.188746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 55/113] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/users/4c56fbc6ee754a96af39128dae8f43a4 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:05.195421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10299c-c9da-4fc4-8837-683351eed6bc None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.235465 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787df478-4b0d-48c7-8632-2d9b6045385a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.236110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787df478-4b0d-48c7-8632-2d9b6045385a None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.236394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787df478-4b0d-48c7-8632-2d9b6045385a None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.237222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787df478-4b0d-48c7-8632-2d9b6045385a None admin] PATH_INFO: `/v3/users/b0b665d93f28427a85c1b52614174fb7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.238384 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787df478-4b0d-48c7-8632-2d9b6045385a None admin] RBAC: Authorizing `identity:get_user(user_id=b0b665d93f28427a85c1b52614174fb7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.241108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787df478-4b0d-48c7-8632-2d9b6045385a None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.247443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 59/114] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/users/b0b665d93f28427a85c1b52614174fb7 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:05.247443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f203f-171b-4068-b9ed-3d135e55cbfd None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.247443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f203f-171b-4068-b9ed-3d135e55cbfd None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.247443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f203f-171b-4068-b9ed-3d135e55cbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.264608 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-5e9f203f-171b-4068-b9ed-3d135e55cbfd None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:05.265825 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-5e9f203f-171b-4068-b9ed-3d135e55cbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:05.321293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10299c-c9da-4fc4-8837-683351eed6bc None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.338674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 60/115] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:05.339376 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f203f-171b-4068-b9ed-3d135e55cbfd None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.365845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33cb22-65e5-40d0-b795-4c3d1f69ffa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.366487 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33cb22-65e5-40d0-b795-4c3d1f69ffa9 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.366800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33cb22-65e5-40d0-b795-4c3d1f69ffa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.367082 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33cb22-65e5-40d0-b795-4c3d1f69ffa9 None admin] PATH_INFO: `/v3/users/3083a658dcab4cb9884523dac0b1b752` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.372080 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:05.381270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33cb22-65e5-40d0-b795-4c3d1f69ffa9 None admin] RBAC: Authorizing `identity:get_user(user_id=3083a658dcab4cb9884523dac0b1b752)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.382761 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33cb22-65e5-40d0-b795-4c3d1f69ffa9 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.384419 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 56/116] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/users/3083a658dcab4cb9884523dac0b1b752 => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:05.385279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33cb22-65e5-40d0-b795-4c3d1f69ffa9 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.401278 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:05.450172 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f203f-171b-4068-b9ed-3d135e55cbfd None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.491625 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1047e-2bce-4f82-838a-8777588f8b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.492272 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1047e-2bce-4f82-838a-8777588f8b4e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.492546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1047e-2bce-4f82-838a-8777588f8b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.492822 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1047e-2bce-4f82-838a-8777588f8b4e None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.493945 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1047e-2bce-4f82-838a-8777588f8b4e None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.495437 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1047e-2bce-4f82-838a-8777588f8b4e None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.497077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33cb22-65e5-40d0-b795-4c3d1f69ffa9 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.497189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 61/117] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:05.497851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1047e-2bce-4f82-838a-8777588f8b4e None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.538583 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de043f0-c020-4096-b653-5bf9b781b97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.539258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de043f0-c020-4096-b653-5bf9b781b97e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.539537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de043f0-c020-4096-b653-5bf9b781b97e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.539921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de043f0-c020-4096-b653-5bf9b781b97e None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.540944 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de043f0-c020-4096-b653-5bf9b781b97e None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.542388 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de043f0-c020-4096-b653-5bf9b781b97e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.549146 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 57/118] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:05.549146 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d8a53-41ee-491a-8d04-d92e3472daf2 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.549146 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d8a53-41ee-491a-8d04-d92e3472daf2 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.549146 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d8a53-41ee-491a-8d04-d92e3472daf2 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.549146 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 58/119] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:05.549146 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d8a53-41ee-491a-8d04-d92e3472daf2 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.609903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:05.614198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1047e-2bce-4f82-838a-8777588f8b4e None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.633751 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:05.648869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9cc90-55cb-45cc-aa8b-c05628bd526f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.649510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9cc90-55cb-45cc-aa8b-c05628bd526f None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.649794 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9cc90-55cb-45cc-aa8b-c05628bd526f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.650072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9cc90-55cb-45cc-aa8b-c05628bd526f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.651224 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9cc90-55cb-45cc-aa8b-c05628bd526f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.652685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9cc90-55cb-45cc-aa8b-c05628bd526f None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.654273 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 62/120] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:05.655044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9cc90-55cb-45cc-aa8b-c05628bd526f None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.657603 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d8a53-41ee-491a-8d04-d92e3472daf2 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.693263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976210d2-e0be-452c-b97b-52ba8b116a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.693876 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976210d2-e0be-452c-b97b-52ba8b116a70 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.694157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976210d2-e0be-452c-b97b-52ba8b116a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.694433 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976210d2-e0be-452c-b97b-52ba8b116a70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.754626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9cc90-55cb-45cc-aa8b-c05628bd526f None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.759619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976210d2-e0be-452c-b97b-52ba8b116a70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.761101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976210d2-e0be-452c-b97b-52ba8b116a70 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.787844 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9434d-3cf9-4fb9-90bf-381ae47ae47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.788511 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9434d-3cf9-4fb9-90bf-381ae47ae47f None admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.788797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9434d-3cf9-4fb9-90bf-381ae47ae47f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.789077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9434d-3cf9-4fb9-90bf-381ae47ae47f None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7/users/4c56fbc6ee754a96af39128dae8f43a4/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.791014 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9434d-3cf9-4fb9-90bf-381ae47ae47f None admin] RBAC: Authorizing `identity:create_grant(project_id=886ef67601844188bb0aa19dcf32dea7, user_id=4c56fbc6ee754a96af39128dae8f43a4, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.792448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd9434d-3cf9-4fb9-90bf-381ae47ae47f None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:05.805178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 63/121] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:05 2022] PUT /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7/users/4c56fbc6ee754a96af39128dae8f43a4/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:05.805967 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9434d-3cf9-4fb9-90bf-381ae47ae47f None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.838398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:05.859479 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:05.892116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 59/122] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:05.892116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500dab4-2226-454e-9414-f74e42100723 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.892116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500dab4-2226-454e-9414-f74e42100723 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.892116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500dab4-2226-454e-9414-f74e42100723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.909478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-0500dab4-2226-454e-9414-f74e42100723 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:05.910790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0500dab4-2226-454e-9414-f74e42100723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:05.947220 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9434d-3cf9-4fb9-90bf-381ae47ae47f None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:05.993585 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f339471-ec12-40d4-9ac3-ced85f55249b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:05.994277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f339471-ec12-40d4-9ac3-ced85f55249b None admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:05.994562 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f339471-ec12-40d4-9ac3-ced85f55249b None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:05.994861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f339471-ec12-40d4-9ac3-ced85f55249b None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7/users/b0b665d93f28427a85c1b52614174fb7/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:05.996832 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f339471-ec12-40d4-9ac3-ced85f55249b None admin] RBAC: Authorizing `identity:create_grant(project_id=886ef67601844188bb0aa19dcf32dea7, user_id=b0b665d93f28427a85c1b52614174fb7, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:05.997419 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 60/123] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:05.998372 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500dab4-2226-454e-9414-f74e42100723 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:05.998495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f339471-ec12-40d4-9ac3-ced85f55249b None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:06.018894 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 64/124] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:05 2022] PUT /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7/users/b0b665d93f28427a85c1b52614174fb7/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:06.018894 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f339471-ec12-40d4-9ac3-ced85f55249b None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:06.183101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500dab4-2226-454e-9414-f74e42100723 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:06.202464 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:06.206891 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:06.227396 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f339471-ec12-40d4-9ac3-ced85f55249b None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:06.255030 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e8945-9a83-4b61-a963-4b1ef8f17e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:06.255973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e8945-9a83-4b61-a963-4b1ef8f17e32 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:06.256436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e8945-9a83-4b61-a963-4b1ef8f17e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:06.256787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e8945-9a83-4b61-a963-4b1ef8f17e32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:06.270159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7e6b3-4ea8-42fa-911f-3990f3935091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:06.270423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e8945-9a83-4b61-a963-4b1ef8f17e32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:06.271163 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e6b3-4ea8-42fa-911f-3990f3935091 None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:06.271583 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e6b3-4ea8-42fa-911f-3990f3935091 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:06.272201 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e6b3-4ea8-42fa-911f-3990f3935091 None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:06.272991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7e6b3-4ea8-42fa-911f-3990f3935091 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:06.274589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7e6b3-4ea8-42fa-911f-3990f3935091 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:06.275891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-4c9e8945-9a83-4b61-a963-4b1ef8f17e32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:06.275891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:06.275891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:06.275891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:06.275891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:06.275891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:06.275891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:06.275891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:06.275891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:06.275891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:06.275891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:06.275891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:06.275891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:06.275891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:06.275891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:06.275891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:06.276909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:06.276909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:06.276909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:06.276909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:06.276909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:06.276909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:06.276909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:06.276909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:06.276909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:06.276909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:06.276909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:06.276909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:06.276909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:06.276909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:06.276909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:06.276909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:06.277841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:06.277841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:06.277841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:06.277841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:06.277841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:06.277841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:06.277841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:06.277841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:06.277841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:06.277841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:06.277841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:06.277841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:06.279512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 61/125] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:06.280390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e8945-9a83-4b61-a963-4b1ef8f17e32 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:06.329112 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 65/126] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Sep 22 14:44:06 2022] POST /identity/v3/projects => generated 302 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:06.330068 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7e6b3-4ea8-42fa-911f-3990f3935091 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:06.452723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e8945-9a83-4b61-a963-4b1ef8f17e32 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:06.525071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0da07f-f98e-45b5-864c-d415605f4d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:06.527127 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0da07f-f98e-45b5-864c-d415605f4d8c None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:06.527408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0da07f-f98e-45b5-864c-d415605f4d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:06.527683 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0da07f-f98e-45b5-864c-d415605f4d8c None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:06.535679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0da07f-f98e-45b5-864c-d415605f4d8c None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:06.537220 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0da07f-f98e-45b5-864c-d415605f4d8c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:06.538077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7e6b3-4ea8-42fa-911f-3990f3935091 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:06.539353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 62/127] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:06 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:06.540180 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0da07f-f98e-45b5-864c-d415605f4d8c None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:06.585612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3466b4-5b3d-463f-8e2b-ff2cc59558ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:06.586451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3466b4-5b3d-463f-8e2b-ff2cc59558ee None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:06.586764 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3466b4-5b3d-463f-8e2b-ff2cc59558ee None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:06.587055 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3466b4-5b3d-463f-8e2b-ff2cc59558ee None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:06.588086 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3466b4-5b3d-463f-8e2b-ff2cc59558ee None admin] RBAC: Adding query filter params (name=member) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:06.588457 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3466b4-5b3d-463f-8e2b-ff2cc59558ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:06.589888 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3466b4-5b3d-463f-8e2b-ff2cc59558ee None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:06.597628 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:06.601865 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 66/128] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:06.610716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3466b4-5b3d-463f-8e2b-ff2cc59558ee None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:06.618571 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:06.702325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0da07f-f98e-45b5-864c-d415605f4d8c None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:06.741494 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e464b-1fd5-4fcc-bce3-76f29f80720b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:06.742421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e464b-1fd5-4fcc-bce3-76f29f80720b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:06.742919 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e464b-1fd5-4fcc-bce3-76f29f80720b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:06.743384 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e464b-1fd5-4fcc-bce3-76f29f80720b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:06.747936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3466b4-5b3d-463f-8e2b-ff2cc59558ee None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:06.756470 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e464b-1fd5-4fcc-bce3-76f29f80720b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:06.761752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-876e464b-1fd5-4fcc-bce3-76f29f80720b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:06.761752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:06.761752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:06.761752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:06.761752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:06.761752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:06.761752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:06.761752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:06.761752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:06.761752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:06.761752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:06.761752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:06.761752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:06.761752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:06.761752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:06.761752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:06.763109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:06.763109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:06.763109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:06.763109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:06.763109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:06.763109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:06.763109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:06.763109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:06.763109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:06.763109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:06.763109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:06.763109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:06.763109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:06.763109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:06.763109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:06.763109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:06.763990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:06.763990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:06.763990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:06.763990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:06.763990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:06.763990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:06.763990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:06.763990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:06.763990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:06.763990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:06.763990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:06.763990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:06.766279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 63/129] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:06.767484 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e464b-1fd5-4fcc-bce3-76f29f80720b None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:06.799721 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b4f00-e9f8-4cd6-aa5e-1f970158bd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:06.801077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b4f00-e9f8-4cd6-aa5e-1f970158bd56 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:06.801565 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b4f00-e9f8-4cd6-aa5e-1f970158bd56 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:06.802096 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b4f00-e9f8-4cd6-aa5e-1f970158bd56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:06.803069 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b4f00-e9f8-4cd6-aa5e-1f970158bd56 None admin] RBAC: Adding query filter params (role.id=77e7991dcd664ba086355ef20ff81541, scope.project.id=886ef67601844188bb0aa19dcf32dea7, user.id=3083a658dcab4cb9884523dac0b1b752) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:06.803595 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b4f00-e9f8-4cd6-aa5e-1f970158bd56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:06.805331 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b4f00-e9f8-4cd6-aa5e-1f970158bd56 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:06.814114 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 67/130] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:06 2022] GET /identity/v3/role_assignments?user.id=3083a658dcab4cb9884523dac0b1b752&scope.project.id=886ef67601844188bb0aa19dcf32dea7&role.id=77e7991dcd664ba086355ef20ff81541 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:06.817168 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b4f00-e9f8-4cd6-aa5e-1f970158bd56 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:06.946635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:06.957678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e464b-1fd5-4fcc-bce3-76f29f80720b None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:06.963109 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:07.005523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05229e5-dff2-4267-94f9-6510463de777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:07.006754 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05229e5-dff2-4267-94f9-6510463de777 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:07.007260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05229e5-dff2-4267-94f9-6510463de777 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:07.007767 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05229e5-dff2-4267-94f9-6510463de777 None admin] PATH_INFO: `/v3/users/b0b665d93f28427a85c1b52614174fb7` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:07.009146 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05229e5-dff2-4267-94f9-6510463de777 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b665d93f28427a85c1b52614174fb7)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:07.010860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05229e5-dff2-4267-94f9-6510463de777 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:07.013086 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 64/131] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:06 2022] GET /identity/v3/users/b0b665d93f28427a85c1b52614174fb7 => generated 275 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:07.019361 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05229e5-dff2-4267-94f9-6510463de777 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:07.040963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b4f00-e9f8-4cd6-aa5e-1f970158bd56 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:07.122358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab60a42-d5d4-43e0-8dfc-ebafcd9d2690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:07.123009 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab60a42-d5d4-43e0-8dfc-ebafcd9d2690 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:07.123295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab60a42-d5d4-43e0-8dfc-ebafcd9d2690 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:07.123574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab60a42-d5d4-43e0-8dfc-ebafcd9d2690 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:07.124266 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab60a42-d5d4-43e0-8dfc-ebafcd9d2690 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:07.124576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab60a42-d5d4-43e0-8dfc-ebafcd9d2690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:07.126006 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab60a42-d5d4-43e0-8dfc-ebafcd9d2690 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:07.144687 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 68/132] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:07.151157 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab60a42-d5d4-43e0-8dfc-ebafcd9d2690 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:07.169158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05229e5-dff2-4267-94f9-6510463de777 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:07.253399 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ecd01-8b04-46d5-a10a-5935561d43b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:07.253968 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ecd01-8b04-46d5-a10a-5935561d43b7 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:07.254251 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ecd01-8b04-46d5-a10a-5935561d43b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:07.254528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ecd01-8b04-46d5-a10a-5935561d43b7 None admin] PATH_INFO: `/v3/projects/1da4a7c776cc4b72afc891ddabce4d99` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:07.255825 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ecd01-8b04-46d5-a10a-5935561d43b7 None admin] RBAC: Authorizing `identity:get_project(project_id=1da4a7c776cc4b72afc891ddabce4d99)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:07.257254 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ecd01-8b04-46d5-a10a-5935561d43b7 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:07.258939 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 65/133] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:07 2022] GET /identity/v3/projects/1da4a7c776cc4b72afc891ddabce4d99 => generated 312 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:07.265337 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ecd01-8b04-46d5-a10a-5935561d43b7 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:07.286511 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab60a42-d5d4-43e0-8dfc-ebafcd9d2690 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:07.327876 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:07.335113 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:07.347174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2751a-738e-449d-9573-b2210ffeebe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:07.347841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2751a-738e-449d-9573-b2210ffeebe0 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:07.348128 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2751a-738e-449d-9573-b2210ffeebe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:07.348403 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2751a-738e-449d-9573-b2210ffeebe0 None admin] PATH_INFO: `/v3/users/b0b665d93f28427a85c1b52614174fb7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:07.353659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2751a-738e-449d-9573-b2210ffeebe0 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b665d93f28427a85c1b52614174fb7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:07.355209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2751a-738e-449d-9573-b2210ffeebe0 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:07.356934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 69/134] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:07 2022] GET /identity/v3/users/b0b665d93f28427a85c1b52614174fb7 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:07.362709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2751a-738e-449d-9573-b2210ffeebe0 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:07.480782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ecd01-8b04-46d5-a10a-5935561d43b7 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:07.548632 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f812ab-3f2d-46ca-9a16-e33c1aaffedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:07.549307 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f812ab-3f2d-46ca-9a16-e33c1aaffedf None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:07.549595 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f812ab-3f2d-46ca-9a16-e33c1aaffedf None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:07.549872 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f812ab-3f2d-46ca-9a16-e33c1aaffedf None admin] PATH_INFO: `/v3/projects/1da4a7c776cc4b72afc891ddabce4d99/users/b0b665d93f28427a85c1b52614174fb7/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:07.556177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f812ab-3f2d-46ca-9a16-e33c1aaffedf None admin] RBAC: Authorizing `identity:create_grant(project_id=1da4a7c776cc4b72afc891ddabce4d99, user_id=b0b665d93f28427a85c1b52614174fb7, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:07.557662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f812ab-3f2d-46ca-9a16-e33c1aaffedf None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:07.572275 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2751a-738e-449d-9573-b2210ffeebe0 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:07.575749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 66/135] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:07 2022] PUT /identity/v3/projects/1da4a7c776cc4b72afc891ddabce4d99/users/b0b665d93f28427a85c1b52614174fb7/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:07.624400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27c37d-07e7-4e55-95d4-9d4986a38836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:07.625060 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27c37d-07e7-4e55-95d4-9d4986a38836 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:07.625349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27c37d-07e7-4e55-95d4-9d4986a38836 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:07.625625 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27c37d-07e7-4e55-95d4-9d4986a38836 None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:07.626809 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27c37d-07e7-4e55-95d4-9d4986a38836 None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:07.628860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27c37d-07e7-4e55-95d4-9d4986a38836 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:07.630512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 70/136] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:07 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:07.636887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f812ab-3f2d-46ca-9a16-e33c1aaffedf None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:07.750200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:07.760846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f812ab-3f2d-46ca-9a16-e33c1aaffedf None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:07.801211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449704b4-5af8-4c66-a9db-875a90c14b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:07.801817 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449704b4-5af8-4c66-a9db-875a90c14b96 None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:07.802090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449704b4-5af8-4c66-a9db-875a90c14b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:07.802373 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449704b4-5af8-4c66-a9db-875a90c14b96 None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7/users/b0b665d93f28427a85c1b52614174fb7/roles/1b632105fdba460b809a2ecd1d186409` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:07.804662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449704b4-5af8-4c66-a9db-875a90c14b96 None admin] RBAC: Authorizing `identity:create_grant(project_id=886ef67601844188bb0aa19dcf32dea7, user_id=b0b665d93f28427a85c1b52614174fb7, role_id=1b632105fdba460b809a2ecd1d186409)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:07.806149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449704b4-5af8-4c66-a9db-875a90c14b96 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:07.821603 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 67/137] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:07 2022] PUT /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7/users/b0b665d93f28427a85c1b52614174fb7/roles/1b632105fdba460b809a2ecd1d186409 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:08.992087 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f87d8-538a-4072-83dd-f46528771ba7 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:08.992627 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f87d8-538a-4072-83dd-f46528771ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:08.992627 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f87d8-538a-4072-83dd-f46528771ba7 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:08.993979 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 71/138] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:44:09.012288 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a38f6f-bc14-4d3a-aecc-736e6ac1c014 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:09.012784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a38f6f-bc14-4d3a-aecc-736e6ac1c014 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:09.012865 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a38f6f-bc14-4d3a-aecc-736e6ac1c014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:09.037309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-92a38f6f-bc14-4d3a-aecc-736e6ac1c014 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:09.038526 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-92a38f6f-bc14-4d3a-aecc-736e6ac1c014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:09.125114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 68/139] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:09.534767 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e451df-84c4-4c59-acfb-c7d696130fe4 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:09.535483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e451df-84c4-4c59-acfb-c7d696130fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:09.536016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e451df-84c4-4c59-acfb-c7d696130fe4 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:09.537575 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 72/140] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:44:09.543257 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b7993-02fc-456e-9851-492f3598073e None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:09.543788 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b7993-02fc-456e-9851-492f3598073e None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:09.544265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b7993-02fc-456e-9851-492f3598073e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:09.565282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-0d2b7993-02fc-456e-9851-492f3598073e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:09.566881 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0d2b7993-02fc-456e-9851-492f3598073e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:09.684240 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e451df-84c4-4c59-acfb-c7d696130fe4 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:09.720891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 69/141] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:09.723647 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f7f22-43a2-4e1e-9166-d0e490cb34df None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:09.724356 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f7f22-43a2-4e1e-9166-d0e490cb34df None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:09.725077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f7f22-43a2-4e1e-9166-d0e490cb34df None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:09.727833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 70/142] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:09.739369 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb40cb-d79e-4ba5-996b-e1ca075bde80 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:09.740163 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb40cb-d79e-4ba5-996b-e1ca075bde80 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:09.740916 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb40cb-d79e-4ba5-996b-e1ca075bde80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:09.782972 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-05fb40cb-d79e-4ba5-996b-e1ca075bde80 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:09.785225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-05fb40cb-d79e-4ba5-996b-e1ca075bde80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:09.805595 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:09.852821 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:09.993114 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e451df-84c4-4c59-acfb-c7d696130fe4 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:10.001097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 71/143] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:10.060660 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e93db-8efc-495b-88ef-32772700b6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:10.061579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e93db-8efc-495b-88ef-32772700b6ce None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.062002 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e93db-8efc-495b-88ef-32772700b6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.062426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e93db-8efc-495b-88ef-32772700b6ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.079607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e93db-8efc-495b-88ef-32772700b6ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:10.085361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-668e93db-8efc-495b-88ef-32772700b6ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:10.085361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:10.085361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:10.085361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:10.085361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:10.085361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:10.085361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:10.085361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:10.085361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:10.085361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:10.085361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:10.085361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:10.085361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:10.085361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:10.085361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:10.085361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:10.086392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:10.086392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:10.086392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:10.086392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:10.086392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:10.086392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:10.086392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:10.086392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:10.086392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:10.086392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:10.086392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:10.086392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:10.086392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:10.086392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:10.086392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:10.086392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:10.087365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:10.087365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:10.087365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:10.087365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:10.087365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:10.087365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:10.087365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:10.087365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:10.087365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:10.087365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:10.087365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:10.087365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:10.089464 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 73/144] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:10.095291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb40cb-d79e-4ba5-996b-e1ca075bde80 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:10.203372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e93db-8efc-495b-88ef-32772700b6ce None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:10.234946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb40cb-d79e-4ba5-996b-e1ca075bde80 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:10.289486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae112e1a-4001-485a-b8eb-8582571ea185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:10.290425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae112e1a-4001-485a-b8eb-8582571ea185 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.290880 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae112e1a-4001-485a-b8eb-8582571ea185 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.291299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae112e1a-4001-485a-b8eb-8582571ea185 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.292108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae112e1a-4001-485a-b8eb-8582571ea185 None admin] RBAC: Adding query filter params (name=member) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:10.292565 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae112e1a-4001-485a-b8eb-8582571ea185 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:10.294140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae112e1a-4001-485a-b8eb-8582571ea185 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:10.303228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 72/145] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:10.309956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af179ef6-7c49-411b-98ca-855d02549388 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.310442 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af179ef6-7c49-411b-98ca-855d02549388 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.310906 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af179ef6-7c49-411b-98ca-855d02549388 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.312444 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 73/146] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:10.313365 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af179ef6-7c49-411b-98ca-855d02549388 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:10.323372 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:10.391998 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e93db-8efc-495b-88ef-32772700b6ce None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:10.433038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:10.454744 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c90b8d-786e-4b44-8315-c9de47b98473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:10.456094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c90b8d-786e-4b44-8315-c9de47b98473 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.456392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c90b8d-786e-4b44-8315-c9de47b98473 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.456671 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c90b8d-786e-4b44-8315-c9de47b98473 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.469719 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c90b8d-786e-4b44-8315-c9de47b98473 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:10.477483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-13c90b8d-786e-4b44-8315-c9de47b98473 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:10.477483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:10.477483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:10.477483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:10.477483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:10.477483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:10.477483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:10.477483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:10.477483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:10.477483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:10.477483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:10.477483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:10.477483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:10.477483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:10.477483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:10.477483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:10.478530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:10.478530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:10.478530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:10.478530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:10.478530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:10.478530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:10.478530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:10.478530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:10.478530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:10.478530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:10.478530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:10.478530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:10.478530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:10.478530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:10.478530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:10.478530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:10.479512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:10.479512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:10.479512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:10.479512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:10.479512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:10.479512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:10.479512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:10.479512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:10.479512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:10.479512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:10.479512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:10.479512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:10.481843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 74/147] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:44:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:10.483385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9cda7-5540-4bf3-b6a1-6b54050907a0 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.483830 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9cda7-5540-4bf3-b6a1-6b54050907a0 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.484254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9cda7-5540-4bf3-b6a1-6b54050907a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.497080 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af179ef6-7c49-411b-98ca-855d02549388 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:10.505873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-d7a9cda7-5540-4bf3-b6a1-6b54050907a0 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:10.507320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d7a9cda7-5540-4bf3-b6a1-6b54050907a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:10.544499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce94f42-6853-4aed-a3ee-304b9584473b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:10.545360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce94f42-6853-4aed-a3ee-304b9584473b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.545883 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce94f42-6853-4aed-a3ee-304b9584473b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.546287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce94f42-6853-4aed-a3ee-304b9584473b None admin] PATH_INFO: `/v3/users/b0b665d93f28427a85c1b52614174fb7` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.547679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce94f42-6853-4aed-a3ee-304b9584473b None admin] RBAC: Authorizing `identity:get_user(user_id=b0b665d93f28427a85c1b52614174fb7)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:10.549952 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce94f42-6853-4aed-a3ee-304b9584473b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:10.552205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 74/148] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:10 2022] GET /identity/v3/users/b0b665d93f28427a85c1b52614174fb7 => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:10.553320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce94f42-6853-4aed-a3ee-304b9584473b None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:10.592362 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 75/149] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:10.593433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9cda7-5540-4bf3-b6a1-6b54050907a0 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:10.630496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:10.663894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce94f42-6853-4aed-a3ee-304b9584473b None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:10.698082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9cda7-5540-4bf3-b6a1-6b54050907a0 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:10.700107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3759b0-f597-48ec-b89b-51fd006654ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:10.700936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3759b0-f597-48ec-b89b-51fd006654ce None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.701356 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3759b0-f597-48ec-b89b-51fd006654ce None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.701775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3759b0-f597-48ec-b89b-51fd006654ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.703091 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3759b0-f597-48ec-b89b-51fd006654ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:10.704697 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3759b0-f597-48ec-b89b-51fd006654ce None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:10.706456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 75/150] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:10.707392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3759b0-f597-48ec-b89b-51fd006654ce None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:10.712245 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:10.735312 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a4f18-7290-47a1-9915-88c20f865194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:10.735940 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a4f18-7290-47a1-9915-88c20f865194 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.736217 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a4f18-7290-47a1-9915-88c20f865194 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.738595 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a4f18-7290-47a1-9915-88c20f865194 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.739331 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a4f18-7290-47a1-9915-88c20f865194 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:10.739640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a4f18-7290-47a1-9915-88c20f865194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:10.741033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a4f18-7290-47a1-9915-88c20f865194 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:10.753002 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 76/151] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:10.754011 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a4f18-7290-47a1-9915-88c20f865194 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:10.852479 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3759b0-f597-48ec-b89b-51fd006654ce None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:10.900505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1081af-3b46-40ee-b801-71ef20f0d3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:10.901132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1081af-3b46-40ee-b801-71ef20f0d3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.901411 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1081af-3b46-40ee-b801-71ef20f0d3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.901683 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1081af-3b46-40ee-b801-71ef20f0d3d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.903177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a4f18-7290-47a1-9915-88c20f865194 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:10.925634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:10.969996 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b2205-dc26-4768-9d1b-49e2fabd1cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:10.970614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b2205-dc26-4768-9d1b-49e2fabd1cb7 None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:10.971880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b2205-dc26-4768-9d1b-49e2fabd1cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:10.972158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b2205-dc26-4768-9d1b-49e2fabd1cb7 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:10.972840 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b2205-dc26-4768-9d1b-49e2fabd1cb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:10.974292 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b2205-dc26-4768-9d1b-49e2fabd1cb7 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:11.011558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 77/152] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:10 2022] POST /identity/v3/users => generated 283 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:11.012430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b2205-dc26-4768-9d1b-49e2fabd1cb7 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:11.048894 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:11.051933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1081af-3b46-40ee-b801-71ef20f0d3d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:11.053436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1081af-3b46-40ee-b801-71ef20f0d3d4 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:11.135437 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b2205-dc26-4768-9d1b-49e2fabd1cb7 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:11.179033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a035b5-0ea2-4061-94e7-8acb2eb57c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:11.179924 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a035b5-0ea2-4061-94e7-8acb2eb57c61 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:11.180480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a035b5-0ea2-4061-94e7-8acb2eb57c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:11.180919 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a035b5-0ea2-4061-94e7-8acb2eb57c61 None admin] PATH_INFO: `/v3/users/4c56fbc6ee754a96af39128dae8f43a4` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:11.182362 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a035b5-0ea2-4061-94e7-8acb2eb57c61 None admin] RBAC: Authorizing `identity:get_user(user_id=4c56fbc6ee754a96af39128dae8f43a4)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:11.184117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a035b5-0ea2-4061-94e7-8acb2eb57c61 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:11.186222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 78/153] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:11 2022] GET /identity/v3/users/4c56fbc6ee754a96af39128dae8f43a4 => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:11.187376 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a035b5-0ea2-4061-94e7-8acb2eb57c61 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:11.266063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 76/154] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:11.271876 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1081af-3b46-40ee-b801-71ef20f0d3d4 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:11.302205 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:11.307127 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a035b5-0ea2-4061-94e7-8acb2eb57c61 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:11.336509 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:11.347529 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ae0c4-fba0-45d1-b3f0-ac076de18cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:11.348177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ae0c4-fba0-45d1-b3f0-ac076de18cce None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:11.348459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ae0c4-fba0-45d1-b3f0-ac076de18cce None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:11.348734 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ae0c4-fba0-45d1-b3f0-ac076de18cce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:11.360885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ae0c4-fba0-45d1-b3f0-ac076de18cce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:11.367886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-6a5ae0c4-fba0-45d1-b3f0-ac076de18cce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:11.367886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:11.367886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:11.367886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:11.367886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:11.367886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:11.367886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:11.367886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:11.367886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:11.367886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:11.367886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:11.367886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:11.367886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:11.367886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:11.367886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:11.367886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:11.368762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:11.368762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:11.368762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:11.368762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:11.368762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:11.368762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:11.368762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:11.368762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:11.368762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:11.368762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:11.368762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:11.368762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:11.368762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:11.368762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:11.368762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:11.368762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:11.369758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:11.369758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:11.369758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:11.369758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:11.369758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:11.369758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:11.369758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:11.369758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:11.369758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:11.369758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:11.369758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:11.369758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:11.369758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 79/155] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:11.370468 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ae0c4-fba0-45d1-b3f0-ac076de18cce None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:11.417025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1081af-3b46-40ee-b801-71ef20f0d3d4 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:11.471730 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02298ebf-bccc-4d44-a9e3-1c6aea10b9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:11.472617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02298ebf-bccc-4d44-a9e3-1c6aea10b9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:11.472897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02298ebf-bccc-4d44-a9e3-1c6aea10b9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:11.473164 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02298ebf-bccc-4d44-a9e3-1c6aea10b9d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:11.521749 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ae0c4-fba0-45d1-b3f0-ac076de18cce None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:11.569119 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02298ebf-bccc-4d44-a9e3-1c6aea10b9d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:11.571090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02298ebf-bccc-4d44-a9e3-1c6aea10b9d3 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:11.573214 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942db86f-22d8-4fd9-baea-5c63dce76971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:11.574189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942db86f-22d8-4fd9-baea-5c63dce76971 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:11.574663 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942db86f-22d8-4fd9-baea-5c63dce76971 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:11.575276 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942db86f-22d8-4fd9-baea-5c63dce76971 None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:11.576523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942db86f-22d8-4fd9-baea-5c63dce76971 None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:11.578111 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942db86f-22d8-4fd9-baea-5c63dce76971 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:11.579989 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 80/156] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:11 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:11.581875 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5cabc-eb46-4e4e-81ad-9dd9f3e3dad4 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:11.582435 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5cabc-eb46-4e4e-81ad-9dd9f3e3dad4 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:11.582710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5cabc-eb46-4e4e-81ad-9dd9f3e3dad4 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:11.583993 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 81/157] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:11.584696 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5cabc-eb46-4e4e-81ad-9dd9f3e3dad4 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:11.622365 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:11.627114 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:11.703695 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 77/158] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:11.705904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1cd56-6591-47d2-954a-f6d1f3e67fc9 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:11.706334 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1cd56-6591-47d2-954a-f6d1f3e67fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:11.706802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1cd56-6591-47d2-954a-f6d1f3e67fc9 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:11.708240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 78/159] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:11.709148 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc1cd56-6591-47d2-954a-f6d1f3e67fc9 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:11.725380 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5cabc-eb46-4e4e-81ad-9dd9f3e3dad4 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:11.764291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0678aa-2c38-4bb9-8234-d69c6a3e87cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:11.765400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0678aa-2c38-4bb9-8234-d69c6a3e87cd None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:11.765873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0678aa-2c38-4bb9-8234-d69c6a3e87cd None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:11.766320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0678aa-2c38-4bb9-8234-d69c6a3e87cd None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:11.767231 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0678aa-2c38-4bb9-8234-d69c6a3e87cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:11.767683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0678aa-2c38-4bb9-8234-d69c6a3e87cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:11.769801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0678aa-2c38-4bb9-8234-d69c6a3e87cd None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:11.779648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 82/160] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:11.781378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f52959-b6bc-434d-8336-a8f7eed2f5a4 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:11.781879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f52959-b6bc-434d-8336-a8f7eed2f5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:11.782146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f52959-b6bc-434d-8336-a8f7eed2f5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:11.804059 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-43f52959-b6bc-434d-8336-a8f7eed2f5a4 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:11.805281 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-43f52959-b6bc-434d-8336-a8f7eed2f5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:11.841854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc1cd56-6591-47d2-954a-f6d1f3e67fc9 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:11.886001 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 83/161] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:11.888282 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248a27b-f828-46ae-84bf-f0e79e109b38 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:11.889305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248a27b-f828-46ae-84bf-f0e79e109b38 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:11.890162 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9744e5-e8a9-47d2-baa1-ad33bef84930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:11.891090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9744e5-e8a9-47d2-baa1-ad33bef84930 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:11.891267 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248a27b-f828-46ae-84bf-f0e79e109b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:11.891737 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9744e5-e8a9-47d2-baa1-ad33bef84930 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:11.892257 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9744e5-e8a9-47d2-baa1-ad33bef84930 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:11.893268 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9744e5-e8a9-47d2-baa1-ad33bef84930 None admin] RBAC: Adding query filter params (role.id=7d3cf58c1ad2407c9f8c60ba99b2006d, scope.project.id=886ef67601844188bb0aa19dcf32dea7, user.id=b0b665d93f28427a85c1b52614174fb7) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:11.893786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9744e5-e8a9-47d2-baa1-ad33bef84930 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:11.895543 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9744e5-e8a9-47d2-baa1-ad33bef84930 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:11.903473 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 79/162] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:11 2022] GET /identity/v3/role_assignments?user.id=b0b665d93f28427a85c1b52614174fb7&scope.project.id=886ef67601844188bb0aa19dcf32dea7&role.id=7d3cf58c1ad2407c9f8c60ba99b2006d => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:11.905115 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9744e5-e8a9-47d2-baa1-ad33bef84930 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:11.909387 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:11.913485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-3248a27b-f828-46ae-84bf-f0e79e109b38 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:11.914844 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3248a27b-f828-46ae-84bf-f0e79e109b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:11.942137 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:12.091325 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 84/163] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:12.092144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248a27b-f828-46ae-84bf-f0e79e109b38 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:12.097806 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9744e5-e8a9-47d2-baa1-ad33bef84930 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:12.169122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e785b-852c-4772-b044-7d6ea86b7d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:12.169836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e785b-852c-4772-b044-7d6ea86b7d11 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:12.170116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e785b-852c-4772-b044-7d6ea86b7d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:12.170383 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e785b-852c-4772-b044-7d6ea86b7d11 None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:12.173304 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e785b-852c-4772-b044-7d6ea86b7d11 None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:12.174832 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e785b-852c-4772-b044-7d6ea86b7d11 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:12.176709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 80/164] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:11 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:12.182307 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e785b-852c-4772-b044-7d6ea86b7d11 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:12.332806 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248a27b-f828-46ae-84bf-f0e79e109b38 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:12.343964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:12.401271 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e785b-852c-4772-b044-7d6ea86b7d11 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:12.409310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee550649-b6c3-43f8-9da4-7407d19ade7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:12.409968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee550649-b6c3-43f8-9da4-7407d19ade7d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:12.410252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee550649-b6c3-43f8-9da4-7407d19ade7d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:12.410520 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee550649-b6c3-43f8-9da4-7407d19ade7d None admin] PATH_INFO: `/v3/users/3083a658dcab4cb9884523dac0b1b752` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:12.411706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee550649-b6c3-43f8-9da4-7407d19ade7d None admin] RBAC: Authorizing `identity:get_user(user_id=3083a658dcab4cb9884523dac0b1b752)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:12.413165 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee550649-b6c3-43f8-9da4-7407d19ade7d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:12.414774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 85/165] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:12 2022] GET /identity/v3/users/3083a658dcab4cb9884523dac0b1b752 => generated 289 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:12.420305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee550649-b6c3-43f8-9da4-7407d19ade7d None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:12.450329 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:12.464650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6ba39-fe93-4885-b62b-72ad3a9462de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:12.465419 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6ba39-fe93-4885-b62b-72ad3a9462de None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:12.465705 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6ba39-fe93-4885-b62b-72ad3a9462de None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:12.465982 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6ba39-fe93-4885-b62b-72ad3a9462de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:12.466724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6ba39-fe93-4885-b62b-72ad3a9462de None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=886ef67601844188bb0aa19dcf32dea7, user.id=4c56fbc6ee754a96af39128dae8f43a4) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:12.467061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6ba39-fe93-4885-b62b-72ad3a9462de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:12.468506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6ba39-fe93-4885-b62b-72ad3a9462de None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:12.480920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 81/166] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:12 2022] GET /identity/v3/role_assignments?user.id=4c56fbc6ee754a96af39128dae8f43a4&scope.project.id=886ef67601844188bb0aa19dcf32dea7&role.id=32c8a81e339a462e973e76bb528ef35a => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:12.521697 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6ba39-fe93-4885-b62b-72ad3a9462de None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:12.589317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee550649-b6c3-43f8-9da4-7407d19ade7d None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:12.654388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f60774-31d6-49ae-bc2b-8b6f42b2a094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:12.654973 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f60774-31d6-49ae-bc2b-8b6f42b2a094 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:12.655255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f60774-31d6-49ae-bc2b-8b6f42b2a094 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:12.655527 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f60774-31d6-49ae-bc2b-8b6f42b2a094 None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:12.657412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f60774-31d6-49ae-bc2b-8b6f42b2a094 None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:12.658932 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f60774-31d6-49ae-bc2b-8b6f42b2a094 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:12.660661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 86/167] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:12 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:12.669097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f60774-31d6-49ae-bc2b-8b6f42b2a094 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:12.748773 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6ba39-fe93-4885-b62b-72ad3a9462de None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:12.808273 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:12.822344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16905384-2120-4305-951a-29358bb6f771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:12.824099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16905384-2120-4305-951a-29358bb6f771 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:12.824099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16905384-2120-4305-951a-29358bb6f771 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:12.824099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16905384-2120-4305-951a-29358bb6f771 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:12.827705 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:12.832293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f60774-31d6-49ae-bc2b-8b6f42b2a094 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:12.849977 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16905384-2120-4305-951a-29358bb6f771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:12.856293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-16905384-2120-4305-951a-29358bb6f771 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:12.856293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:12.856293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:12.856293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:12.856293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:12.856293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:12.856293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:12.856293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:12.856293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:12.856293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:12.856293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:12.856293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:12.856293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:12.856293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:12.856293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:12.856293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:12.857449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:12.857449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:12.857449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:12.857449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:12.857449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:12.857449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:12.857449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:12.857449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:12.857449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:12.857449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:12.857449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:12.857449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:12.857449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:12.857449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:12.857449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:12.857449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:12.858247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:12.858247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:12.858247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:12.858247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:12.858247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:12.858247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:12.858247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:12.858247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:12.858247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:12.858247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:12.858247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:12.858247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:12.860314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 82/168] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:12.865989 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16905384-2120-4305-951a-29358bb6f771 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:12.872322 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813fcf48-e4a8-4224-b421-6e8bcde6b67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:12.873060 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fcf48-e4a8-4224-b421-6e8bcde6b67d None admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:12.873340 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fcf48-e4a8-4224-b421-6e8bcde6b67d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:12.873612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813fcf48-e4a8-4224-b421-6e8bcde6b67d None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7/users/3083a658dcab4cb9884523dac0b1b752/roles/77e7991dcd664ba086355ef20ff81541` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:12.875734 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813fcf48-e4a8-4224-b421-6e8bcde6b67d None admin] RBAC: Authorizing `identity:create_grant(project_id=886ef67601844188bb0aa19dcf32dea7, user_id=3083a658dcab4cb9884523dac0b1b752, role_id=77e7991dcd664ba086355ef20ff81541)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:12.877740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813fcf48-e4a8-4224-b421-6e8bcde6b67d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:12.893323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 87/169] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:12 2022] PUT /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7/users/3083a658dcab4cb9884523dac0b1b752/roles/77e7991dcd664ba086355ef20ff81541 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:12.894200 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813fcf48-e4a8-4224-b421-6e8bcde6b67d None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:13.036287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16905384-2120-4305-951a-29358bb6f771 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:13.041738 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813fcf48-e4a8-4224-b421-6e8bcde6b67d None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:13.083851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac826278-d967-4401-b5cb-d1237f7ac0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:13.084502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac826278-d967-4401-b5cb-d1237f7ac0dc None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:13.084783 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac826278-d967-4401-b5cb-d1237f7ac0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:13.085059 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac826278-d967-4401-b5cb-d1237f7ac0dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:13.092024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f870c0a-f1d5-4a3f-8532-d6b9f40e9b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:13.092658 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f870c0a-f1d5-4a3f-8532-d6b9f40e9b3f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:13.092940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f870c0a-f1d5-4a3f-8532-d6b9f40e9b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:13.093209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f870c0a-f1d5-4a3f-8532-d6b9f40e9b3f None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:13.093892 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f870c0a-f1d5-4a3f-8532-d6b9f40e9b3f None admin] RBAC: Adding query filter params (name=member) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:13.094199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f870c0a-f1d5-4a3f-8532-d6b9f40e9b3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:13.095640 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f870c0a-f1d5-4a3f-8532-d6b9f40e9b3f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:13.112262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac826278-d967-4401-b5cb-d1237f7ac0dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:13.117386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-ac826278-d967-4401-b5cb-d1237f7ac0dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:13.117386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:13.117386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:13.117386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:13.117386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:13.117386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:13.117386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:13.117386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:13.117386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:13.117386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:13.117386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:13.117386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:13.117386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:13.117386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:13.117386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:13.117386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:13.118290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:13.118290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:13.118290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:13.118290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:13.118290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:13.118290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:13.118290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:13.118290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:13.118290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:13.118290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:13.118290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:13.118290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:13.118290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:13.118290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:13.118290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:13.118290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:13.119197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:13.119197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:13.119197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:13.119197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:13.119197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:13.119197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:13.119197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:13.119197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:13.119197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:13.119197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:13.119197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:13.119197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:13.119197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 88/170] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:13.124612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac826278-d967-4401-b5cb-d1237f7ac0dc None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:13.127988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 83/171] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:13.129828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:13.134385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f870c0a-f1d5-4a3f-8532-d6b9f40e9b3f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:13.246599 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac826278-d967-4401-b5cb-d1237f7ac0dc None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:13.264353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:13.294877 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887104b-8c5f-4809-a7e9-fed3297a2256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:13.295424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887104b-8c5f-4809-a7e9-fed3297a2256 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:13.295706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887104b-8c5f-4809-a7e9-fed3297a2256 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:13.295982 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887104b-8c5f-4809-a7e9-fed3297a2256 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:13.296781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887104b-8c5f-4809-a7e9-fed3297a2256 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:13.297093 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887104b-8c5f-4809-a7e9-fed3297a2256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:13.298481 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7887104b-8c5f-4809-a7e9-fed3297a2256 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:13.307959 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 89/172] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:13.315787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887104b-8c5f-4809-a7e9-fed3297a2256 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:13.317470 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f870c0a-f1d5-4a3f-8532-d6b9f40e9b3f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:13.368910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a6d52-c25a-49a7-8715-4a6a0ecf7703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:13.369759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a6d52-c25a-49a7-8715-4a6a0ecf7703 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:13.370662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a6d52-c25a-49a7-8715-4a6a0ecf7703 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:13.371113 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a6d52-c25a-49a7-8715-4a6a0ecf7703 None admin] PATH_INFO: `/v3/users/b0b665d93f28427a85c1b52614174fb7` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:13.372835 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a6d52-c25a-49a7-8715-4a6a0ecf7703 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b665d93f28427a85c1b52614174fb7)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:13.374827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a6d52-c25a-49a7-8715-4a6a0ecf7703 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:13.376858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 84/173] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:13 2022] GET /identity/v3/users/b0b665d93f28427a85c1b52614174fb7 => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:13.382826 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a6d52-c25a-49a7-8715-4a6a0ecf7703 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:13.407805 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:13.447215 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887104b-8c5f-4809-a7e9-fed3297a2256 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:13.488730 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ef5f8-70bf-415a-bf92-eaf856335f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:13.490052 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ef5f8-70bf-415a-bf92-eaf856335f39 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:13.490409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ef5f8-70bf-415a-bf92-eaf856335f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:13.490793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ef5f8-70bf-415a-bf92-eaf856335f39 None admin] PATH_INFO: `/v3/users/b0b665d93f28427a85c1b52614174fb7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:13.492061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ef5f8-70bf-415a-bf92-eaf856335f39 None admin] RBAC: Authorizing `identity:get_user(user_id=b0b665d93f28427a85c1b52614174fb7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:13.493813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ef5f8-70bf-415a-bf92-eaf856335f39 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:13.496545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 90/174] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:13 2022] GET /identity/v3/users/b0b665d93f28427a85c1b52614174fb7 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:13.502102 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ef5f8-70bf-415a-bf92-eaf856335f39 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:13.530136 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a6d52-c25a-49a7-8715-4a6a0ecf7703 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:13.584591 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:13.603178 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de19207-b78a-46c5-8f0f-78a3ba6cf6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:13.603807 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de19207-b78a-46c5-8f0f-78a3ba6cf6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:13.604079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de19207-b78a-46c5-8f0f-78a3ba6cf6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:13.604344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de19207-b78a-46c5-8f0f-78a3ba6cf6f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:13.710317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ef5f8-70bf-415a-bf92-eaf856335f39 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:13.748625 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de19207-b78a-46c5-8f0f-78a3ba6cf6f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:13.750443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de19207-b78a-46c5-8f0f-78a3ba6cf6f0 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:13.761499 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b7c73-5fa6-475e-9ce7-283de0f1f701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:13.762116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b7c73-5fa6-475e-9ce7-283de0f1f701 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:13.762392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b7c73-5fa6-475e-9ce7-283de0f1f701 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:13.762663 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b7c73-5fa6-475e-9ce7-283de0f1f701 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:13.772839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:13.844438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b7c73-5fa6-475e-9ce7-283de0f1f701 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:13.846841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b7c73-5fa6-475e-9ce7-283de0f1f701 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:13.958264 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 91/175] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:13.972644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b7c73-5fa6-475e-9ce7-283de0f1f701 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:13.976063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 85/176] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:13.986331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de19207-b78a-46c5-8f0f-78a3ba6cf6f0 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:14.013005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:14.023401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:14.087388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b7c73-5fa6-475e-9ce7-283de0f1f701 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:14.104032 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de19207-b78a-46c5-8f0f-78a3ba6cf6f0 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:14.124510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190d83f-5e83-4396-9fa6-52778f141769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:14.125229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190d83f-5e83-4396-9fa6-52778f141769 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:14.125421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190d83f-5e83-4396-9fa6-52778f141769 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:14.125696 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190d83f-5e83-4396-9fa6-52778f141769 None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:14.126918 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190d83f-5e83-4396-9fa6-52778f141769 None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:14.128743 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190d83f-5e83-4396-9fa6-52778f141769 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:14.131248 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 92/177] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:13 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:14.137916 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190d83f-5e83-4396-9fa6-52778f141769 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:14.175318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7b59f-b875-4aa8-850d-949d87a8e475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:14.176250 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7b59f-b875-4aa8-850d-949d87a8e475 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:14.176681 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7b59f-b875-4aa8-850d-949d87a8e475 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:14.177163 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7b59f-b875-4aa8-850d-949d87a8e475 None admin] PATH_INFO: `/v3/projects/1da4a7c776cc4b72afc891ddabce4d99` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:14.183667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7b59f-b875-4aa8-850d-949d87a8e475 None admin] RBAC: Authorizing `identity:get_project(project_id=1da4a7c776cc4b72afc891ddabce4d99)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:14.185688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7b59f-b875-4aa8-850d-949d87a8e475 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:14.187970 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 86/178] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:13 2022] GET /identity/v3/projects/1da4a7c776cc4b72afc891ddabce4d99 => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:14.207757 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7b59f-b875-4aa8-850d-949d87a8e475 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:14.285526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190d83f-5e83-4396-9fa6-52778f141769 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:14.307751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:14.325569 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f15c4-8005-4d12-86be-f0843eac53c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:14.326193 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f15c4-8005-4d12-86be-f0843eac53c7 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:14.326475 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f15c4-8005-4d12-86be-f0843eac53c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:14.326764 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f15c4-8005-4d12-86be-f0843eac53c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:14.327501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f15c4-8005-4d12-86be-f0843eac53c7 None admin] RBAC: Adding query filter params (role.id=1b632105fdba460b809a2ecd1d186409, scope.project.id=886ef67601844188bb0aa19dcf32dea7, user.id=b0b665d93f28427a85c1b52614174fb7) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:14.327928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f15c4-8005-4d12-86be-f0843eac53c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:14.329505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41f15c4-8005-4d12-86be-f0843eac53c7 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:14.337041 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 93/179] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:14 2022] GET /identity/v3/role_assignments?user.id=b0b665d93f28427a85c1b52614174fb7&scope.project.id=886ef67601844188bb0aa19dcf32dea7&role.id=1b632105fdba460b809a2ecd1d186409 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:14.403142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:14.415768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7b59f-b875-4aa8-850d-949d87a8e475 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:14.482381 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c70fc-33c0-465f-bc00-ea0348cc5378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:14.483047 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c70fc-33c0-465f-bc00-ea0348cc5378 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:14.483324 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c70fc-33c0-465f-bc00-ea0348cc5378 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:14.483597 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c70fc-33c0-465f-bc00-ea0348cc5378 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:14.484343 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c70fc-33c0-465f-bc00-ea0348cc5378 None admin] RBAC: Adding query filter params (role.id=7d3cf58c1ad2407c9f8c60ba99b2006d, scope.project.id=1da4a7c776cc4b72afc891ddabce4d99, user.id=b0b665d93f28427a85c1b52614174fb7) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:14.484650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c70fc-33c0-465f-bc00-ea0348cc5378 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:14.486137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c70fc-33c0-465f-bc00-ea0348cc5378 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:14.495852 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 87/180] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:14 2022] GET /identity/v3/role_assignments?user.id=b0b665d93f28427a85c1b52614174fb7&scope.project.id=1da4a7c776cc4b72afc891ddabce4d99&role.id=7d3cf58c1ad2407c9f8c60ba99b2006d => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:14.769547 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d6315-9663-4078-9ff9-b47e2bb353f3 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:14.770202 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d6315-9663-4078-9ff9-b47e2bb353f3 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:14.770703 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d6315-9663-4078-9ff9-b47e2bb353f3 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:14.772297 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 94/181] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:14.779247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e254a72-357b-4df0-917a-ae1f55b16127 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:14.779832 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e254a72-357b-4df0-917a-ae1f55b16127 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:14.780301 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e254a72-357b-4df0-917a-ae1f55b16127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:14.800251 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-3e254a72-357b-4df0-917a-ae1f55b16127 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:14.801880 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3e254a72-357b-4df0-917a-ae1f55b16127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:14.890926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 88/182] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:15.020044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8db4e-1ace-41a0-9c99-12bed4900955 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.020561 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8db4e-1ace-41a0-9c99-12bed4900955 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.020837 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8db4e-1ace-41a0-9c99-12bed4900955 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.022329 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 95/183] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:15.028033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd06cd1-702c-46c4-85ef-4b7616a27da9 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.028362 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd06cd1-702c-46c4-85ef-4b7616a27da9 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.028599 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd06cd1-702c-46c4-85ef-4b7616a27da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.053068 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-cbd06cd1-702c-46c4-85ef-4b7616a27da9 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:15.054253 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-cbd06cd1-702c-46c4-85ef-4b7616a27da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:15.133270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 89/184] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:15.242674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8db4e-1ace-41a0-9c99-12bed4900955 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:15.374669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8db4e-1ace-41a0-9c99-12bed4900955 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:15.414438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e704f66-f29c-42fe-8021-b1ce2f717f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:15.415149 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704f66-f29c-42fe-8021-b1ce2f717f14 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.415429 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704f66-f29c-42fe-8021-b1ce2f717f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.415706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704f66-f29c-42fe-8021-b1ce2f717f14 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.426397 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e704f66-f29c-42fe-8021-b1ce2f717f14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:15.431793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-0e704f66-f29c-42fe-8021-b1ce2f717f14 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:15.431793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:15.431793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:15.431793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:15.431793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:15.431793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:15.431793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:15.431793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:15.431793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:15.431793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:15.431793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:15.431793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:15.431793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:15.431793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:15.431793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:15.431793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:15.432668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:15.432668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:15.432668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:15.432668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:15.432668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:15.432668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:15.432668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:15.432668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:15.432668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:15.432668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:15.432668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:15.432668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:15.432668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:15.432668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:15.432668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:15.432668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:15.433469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:15.433469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:15.433469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:15.433469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:15.433469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:15.433469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:15.433469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:15.433469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:15.433469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:15.433469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:15.433469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:15.433469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:15.433469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 96/185] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:15.439060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd06cd1-702c-46c4-85ef-4b7616a27da9 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:15.459607 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:15.468797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e704f66-f29c-42fe-8021-b1ce2f717f14 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:15.502877 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:15.555129 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd06cd1-702c-46c4-85ef-4b7616a27da9 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:15.587509 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e704f66-f29c-42fe-8021-b1ce2f717f14 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:15.595140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca122d1-ae9a-48ec-aa12-0253fc1fc3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:15.596027 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca122d1-ae9a-48ec-aa12-0253fc1fc3db None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.596313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca122d1-ae9a-48ec-aa12-0253fc1fc3db None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.596584 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca122d1-ae9a-48ec-aa12-0253fc1fc3db None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.597264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca122d1-ae9a-48ec-aa12-0253fc1fc3db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:15.597569 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca122d1-ae9a-48ec-aa12-0253fc1fc3db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:15.598969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca122d1-ae9a-48ec-aa12-0253fc1fc3db None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:15.608183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 90/186] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:15.609237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a8516-6c7b-44a7-886c-7caadb4bc2c4 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.609657 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a8516-6c7b-44a7-886c-7caadb4bc2c4 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.609936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a8516-6c7b-44a7-886c-7caadb4bc2c4 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.611302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 91/187] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:15.616502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a8516-6c7b-44a7-886c-7caadb4bc2c4 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:15.636577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e69337-1ed9-466e-bca4-37379be0f4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:15.637210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e69337-1ed9-466e-bca4-37379be0f4cd None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.637486 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e69337-1ed9-466e-bca4-37379be0f4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.637755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e69337-1ed9-466e-bca4-37379be0f4cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.648878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e69337-1ed9-466e-bca4-37379be0f4cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:15.654198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-f0e69337-1ed9-466e-bca4-37379be0f4cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:15.654198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:15.654198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:15.654198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:15.654198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:15.654198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:15.654198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:15.654198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:15.654198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:15.654198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:15.654198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:15.654198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:15.654198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:15.654198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:15.654198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:15.654198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:15.655293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:15.655293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:15.655293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:15.655293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:15.655293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:15.655293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:15.655293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:15.655293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:15.655293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:15.655293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:15.655293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:15.655293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:15.655293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:15.655293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:15.655293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:15.655293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:15.656277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:15.656277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:15.656277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:15.656277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:15.656277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:15.656277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:15.656277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:15.656277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:15.656277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:15.656277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:15.656277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:15.656277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:15.656277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 97/188] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:44:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:15.657094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4957b-bdc5-4891-91c4-9700ecfc3923 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.657094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4957b-bdc5-4891-91c4-9700ecfc3923 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.657094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4957b-bdc5-4891-91c4-9700ecfc3923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.679849 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-87e4957b-bdc5-4891-91c4-9700ecfc3923 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:15.681074 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-87e4957b-bdc5-4891-91c4-9700ecfc3923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:15.706365 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:15.728207 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a8516-6c7b-44a7-886c-7caadb4bc2c4 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:15.762028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 98/189] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:15.763136 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4957b-bdc5-4891-91c4-9700ecfc3923 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:15.774073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6892e-e4d9-4e48-9ea3-36eb6d9152d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:15.775039 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6892e-e4d9-4e48-9ea3-36eb6d9152d0 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.775564 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6892e-e4d9-4e48-9ea3-36eb6d9152d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.776033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6892e-e4d9-4e48-9ea3-36eb6d9152d0 None admin] PATH_INFO: `/v3/users/fe2049aff63c4d76a84c0a2cb92bb60b` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.788121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:15.795926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6892e-e4d9-4e48-9ea3-36eb6d9152d0 None admin] RBAC: Authorizing `identity:get_user(user_id=fe2049aff63c4d76a84c0a2cb92bb60b)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:15.798639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d6892e-e4d9-4e48-9ea3-36eb6d9152d0 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:15.802210 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 92/190] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:15 2022] GET /identity/v3/users/fe2049aff63c4d76a84c0a2cb92bb60b => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:15.807676 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6892e-e4d9-4e48-9ea3-36eb6d9152d0 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:15.883186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4957b-bdc5-4891-91c4-9700ecfc3923 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:15.911433 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6892e-e4d9-4e48-9ea3-36eb6d9152d0 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:15.923617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e3b14-3b82-4f6d-96f6-05d726b8777a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:15.924748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e3b14-3b82-4f6d-96f6-05d726b8777a None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.925036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e3b14-3b82-4f6d-96f6-05d726b8777a None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.925304 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e3b14-3b82-4f6d-96f6-05d726b8777a None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.925965 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e3b14-3b82-4f6d-96f6-05d726b8777a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:15.926268 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e3b14-3b82-4f6d-96f6-05d726b8777a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:15.927674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e3b14-3b82-4f6d-96f6-05d726b8777a None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:15.936595 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 99/191] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:15.942682 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c78d5-7818-4579-8178-f00a9cf519c2 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.942989 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c78d5-7818-4579-8178-f00a9cf519c2 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.943262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c78d5-7818-4579-8178-f00a9cf519c2 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.946882 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 100/192] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:15.946882 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c78d5-7818-4579-8178-f00a9cf519c2 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:15.950110 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6c154-eaba-44cb-90d2-d4d32441a1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:15.950720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6c154-eaba-44cb-90d2-d4d32441a1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:15.951009 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6c154-eaba-44cb-90d2-d4d32441a1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:15.951275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6c154-eaba-44cb-90d2-d4d32441a1b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:15.960139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:16.026039 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b6c154-eaba-44cb-90d2-d4d32441a1b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:16.027620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b6c154-eaba-44cb-90d2-d4d32441a1b8 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:16.043463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:16.068452 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c78d5-7818-4579-8178-f00a9cf519c2 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:16.102630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefceb4-2c69-4496-8c35-fddbfd5e2741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:16.103287 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefceb4-2c69-4496-8c35-fddbfd5e2741 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:16.103562 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefceb4-2c69-4496-8c35-fddbfd5e2741 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:16.103832 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefceb4-2c69-4496-8c35-fddbfd5e2741 None admin] PATH_INFO: `/v3/users/fe2049aff63c4d76a84c0a2cb92bb60b` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:16.104955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefceb4-2c69-4496-8c35-fddbfd5e2741 None admin] RBAC: Authorizing `identity:get_user(user_id=fe2049aff63c4d76a84c0a2cb92bb60b)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:16.106382 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefceb4-2c69-4496-8c35-fddbfd5e2741 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:16.107989 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 101/193] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:15 2022] GET /identity/v3/users/fe2049aff63c4d76a84c0a2cb92bb60b => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:16.109276 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4c0a2-3b92-492d-8c2d-03584618bbbb None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:16.109556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4c0a2-3b92-492d-8c2d-03584618bbbb None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:16.109822 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4c0a2-3b92-492d-8c2d-03584618bbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:16.127357 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-75d4c0a2-3b92-492d-8c2d-03584618bbbb None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:16.128534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-75d4c0a2-3b92-492d-8c2d-03584618bbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:16.135520 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 93/194] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:16.136307 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6c154-eaba-44cb-90d2-d4d32441a1b8 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:16.177409 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:16.194603 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 102/195] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:16.195814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4c0a2-3b92-492d-8c2d-03584618bbbb None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:16.249448 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6c154-eaba-44cb-90d2-d4d32441a1b8 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:16.275836 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:16.285942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc5035-e00c-407c-ad24-18afb35c18ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:16.286583 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc5035-e00c-407c-ad24-18afb35c18ab None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:16.286884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc5035-e00c-407c-ad24-18afb35c18ab None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:16.287191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc5035-e00c-407c-ad24-18afb35c18ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:16.288958 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc5035-e00c-407c-ad24-18afb35c18ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:16.290567 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc5035-e00c-407c-ad24-18afb35c18ab None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:16.296591 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 94/196] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:16.296591 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc5035-e00c-407c-ad24-18afb35c18ab None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:16.304527 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4c0a2-3b92-492d-8c2d-03584618bbbb None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:16.344611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c8caa-a939-4e77-a247-08480756136c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:16.345273 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c8caa-a939-4e77-a247-08480756136c None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:16.345494 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c8caa-a939-4e77-a247-08480756136c None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:16.346777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c8caa-a939-4e77-a247-08480756136c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:16.406230 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc5035-e00c-407c-ad24-18afb35c18ab None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:16.414095 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:16.423755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c8caa-a939-4e77-a247-08480756136c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:16.425249 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c8caa-a939-4e77-a247-08480756136c None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:16.443056 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8df2b5-8550-473a-b30b-76fb23a5fb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:16.443784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8df2b5-8550-473a-b30b-76fb23a5fb14 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:16.444164 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8df2b5-8550-473a-b30b-76fb23a5fb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:16.444330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8df2b5-8550-473a-b30b-76fb23a5fb14 None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:16.445655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8df2b5-8550-473a-b30b-76fb23a5fb14 None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:16.447107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8df2b5-8550-473a-b30b-76fb23a5fb14 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:16.448748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 95/197] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:16 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:16.449487 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8df2b5-8550-473a-b30b-76fb23a5fb14 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:16.507936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:16.532906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 103/198] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:16.532906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c8caa-a939-4e77-a247-08480756136c None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:16.553847 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8df2b5-8550-473a-b30b-76fb23a5fb14 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:16.588663 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3b526-1870-4a88-acb9-4b0ae77a001c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:16.589273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3b526-1870-4a88-acb9-4b0ae77a001c None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:16.589550 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3b526-1870-4a88-acb9-4b0ae77a001c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:16.589821 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3b526-1870-4a88-acb9-4b0ae77a001c None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:16.590472 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3b526-1870-4a88-acb9-4b0ae77a001c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:16.591965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3b526-1870-4a88-acb9-4b0ae77a001c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:16.615871 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 96/199] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:16 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:16.616693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3b526-1870-4a88-acb9-4b0ae77a001c None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:16.637378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c8caa-a939-4e77-a247-08480756136c None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:16.649982 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:16.680709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e43f0-ab57-4877-a364-ffdc9a37e948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:16.681418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e43f0-ab57-4877-a364-ffdc9a37e948 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:16.682128 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e43f0-ab57-4877-a364-ffdc9a37e948 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:16.682406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e43f0-ab57-4877-a364-ffdc9a37e948 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:16.683348 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e43f0-ab57-4877-a364-ffdc9a37e948 None admin] RBAC: Adding query filter params (role.id=1b632105fdba460b809a2ecd1d186409, scope.project.id=45e79ea1241346c58bf04c1494b3dc7d, user.id=fe2049aff63c4d76a84c0a2cb92bb60b) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:16.683662 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e43f0-ab57-4877-a364-ffdc9a37e948 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:16.685142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e43f0-ab57-4877-a364-ffdc9a37e948 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:16.692435 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 104/200] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:16 2022] GET /identity/v3/role_assignments?user.id=fe2049aff63c4d76a84c0a2cb92bb60b&scope.project.id=45e79ea1241346c58bf04c1494b3dc7d&role.id=1b632105fdba460b809a2ecd1d186409 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:16.693117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e43f0-ab57-4877-a364-ffdc9a37e948 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:16.748494 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:16.750772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3b526-1870-4a88-acb9-4b0ae77a001c None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:16.795760 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc51ab1-af8b-4771-9356-1f245ef7a419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:16.796702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc51ab1-af8b-4771-9356-1f245ef7a419 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:16.797298 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc51ab1-af8b-4771-9356-1f245ef7a419 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:16.797752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc51ab1-af8b-4771-9356-1f245ef7a419 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:16.811758 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc51ab1-af8b-4771-9356-1f245ef7a419 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:16.819486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-0dc51ab1-af8b-4771-9356-1f245ef7a419 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:16.819486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:16.819486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:16.819486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:16.819486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:16.819486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:16.819486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:16.819486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:16.819486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:16.819486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:16.819486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:16.819486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:16.819486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:16.819486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:16.819486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:16.819486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:16.820505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:16.820505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:16.820505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:16.820505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:16.820505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:16.820505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:16.820505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:16.820505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:16.820505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:16.820505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:16.820505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:16.820505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:16.820505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:16.820505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:16.820505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:16.820505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:16.821302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:16.821302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:16.821302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:16.821302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:16.821302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:16.821302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:16.821302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:16.821302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:16.821302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:16.821302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:16.821302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:16.821302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:16.823672 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 97/201] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:16.827773 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e43f0-ab57-4877-a364-ffdc9a37e948 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:16.829354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc51ab1-af8b-4771-9356-1f245ef7a419 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:16.887677 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515440bd-a78f-4257-bb4f-ecc0372255de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:16.888325 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515440bd-a78f-4257-bb4f-ecc0372255de None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:16.888598 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515440bd-a78f-4257-bb4f-ecc0372255de None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:16.888983 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515440bd-a78f-4257-bb4f-ecc0372255de None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:16.892166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515440bd-a78f-4257-bb4f-ecc0372255de None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:16.895507 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515440bd-a78f-4257-bb4f-ecc0372255de None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:16.897543 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 105/202] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:16 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:16.903694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515440bd-a78f-4257-bb4f-ecc0372255de None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:16.953239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:16.965273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc51ab1-af8b-4771-9356-1f245ef7a419 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:17.010772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebe538-891b-4d00-9899-cc0023966d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:17.011924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebe538-891b-4d00-9899-cc0023966d8d None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:17.012211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebe538-891b-4d00-9899-cc0023966d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:17.012484 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebe538-891b-4d00-9899-cc0023966d8d None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:17.013325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebe538-891b-4d00-9899-cc0023966d8d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:17.013637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebe538-891b-4d00-9899-cc0023966d8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:17.015071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebe538-891b-4d00-9899-cc0023966d8d None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:17.024400 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 98/203] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:17.030103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebe538-891b-4d00-9899-cc0023966d8d None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:17.035896 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515440bd-a78f-4257-bb4f-ecc0372255de None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:17.047476 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:17.078959 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a88c5-6e63-431b-8166-4db03598cd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:17.081156 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a88c5-6e63-431b-8166-4db03598cd20 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:17.081693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a88c5-6e63-431b-8166-4db03598cd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:17.082104 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a88c5-6e63-431b-8166-4db03598cd20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:17.083061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a88c5-6e63-431b-8166-4db03598cd20 None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=45e79ea1241346c58bf04c1494b3dc7d, user.id=fe2049aff63c4d76a84c0a2cb92bb60b) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:17.083506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a88c5-6e63-431b-8166-4db03598cd20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:17.085103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a88c5-6e63-431b-8166-4db03598cd20 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:17.092754 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 106/204] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:16 2022] GET /identity/v3/role_assignments?user.id=fe2049aff63c4d76a84c0a2cb92bb60b&scope.project.id=45e79ea1241346c58bf04c1494b3dc7d&role.id=32c8a81e339a462e973e76bb528ef35a => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:17.162980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebe538-891b-4d00-9899-cc0023966d8d None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:17.223093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457ac56-b5c4-4359-b95a-043a197dffd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:17.223970 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457ac56-b5c4-4359-b95a-043a197dffd0 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:17.224386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457ac56-b5c4-4359-b95a-043a197dffd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:17.224810 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457ac56-b5c4-4359-b95a-043a197dffd0 None admin] PATH_INFO: `/v3/users/3083a658dcab4cb9884523dac0b1b752` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:17.231700 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457ac56-b5c4-4359-b95a-043a197dffd0 None admin] RBAC: Authorizing `identity:get_user(user_id=3083a658dcab4cb9884523dac0b1b752)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:17.233428 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457ac56-b5c4-4359-b95a-043a197dffd0 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:17.235731 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 99/205] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:17 2022] GET /identity/v3/users/3083a658dcab4cb9884523dac0b1b752 => generated 289 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:17.241612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a88c5-6e63-431b-8166-4db03598cd20 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:17.369016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a88c5-6e63-431b-8166-4db03598cd20 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:17.404466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c43f5-0869-43f0-a6ea-92e532f1dc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:17.405071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c43f5-0869-43f0-a6ea-92e532f1dc3a None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:17.405349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c43f5-0869-43f0-a6ea-92e532f1dc3a None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:17.405617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c43f5-0869-43f0-a6ea-92e532f1dc3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:17.449343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:17.476714 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59c43f5-0869-43f0-a6ea-92e532f1dc3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:17.478113 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59c43f5-0869-43f0-a6ea-92e532f1dc3a None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:17.579596 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 107/206] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:17.585748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457ac56-b5c4-4359-b95a-043a197dffd0 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:17.608624 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:17.719507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457ac56-b5c4-4359-b95a-043a197dffd0 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:17.773319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19050731-f019-4e2c-a30e-b7750b70d169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:17.773871 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19050731-f019-4e2c-a30e-b7750b70d169 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:17.774145 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19050731-f019-4e2c-a30e-b7750b70d169 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:17.774415 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19050731-f019-4e2c-a30e-b7750b70d169 None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:17.775570 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19050731-f019-4e2c-a30e-b7750b70d169 None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:17.777018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19050731-f019-4e2c-a30e-b7750b70d169 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:17.778654 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 100/207] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:17 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:17.785056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c43f5-0869-43f0-a6ea-92e532f1dc3a None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:17.868977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:17.900924 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c43f5-0869-43f0-a6ea-92e532f1dc3a None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:17.938858 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6123a-6773-497e-bfb9-18c243436c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:17.939489 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6123a-6773-497e-bfb9-18c243436c8e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:17.940349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6123a-6773-497e-bfb9-18c243436c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:17.940661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6123a-6773-497e-bfb9-18c243436c8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:17.941400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6123a-6773-497e-bfb9-18c243436c8e None admin] RBAC: Adding query filter params (role.id=77e7991dcd664ba086355ef20ff81541, scope.project.id=886ef67601844188bb0aa19dcf32dea7, user.id=3083a658dcab4cb9884523dac0b1b752) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:17.941696 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6123a-6773-497e-bfb9-18c243436c8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:17.943209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6123a-6773-497e-bfb9-18c243436c8e None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:17.949466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 108/208] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:17 2022] GET /identity/v3/role_assignments?user.id=3083a658dcab4cb9884523dac0b1b752&scope.project.id=886ef67601844188bb0aa19dcf32dea7&role.id=77e7991dcd664ba086355ef20ff81541 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:19.157039 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dffd32-06a0-4a7b-9a7e-5d61b130f08b None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.157571 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dffd32-06a0-4a7b-9a7e-5d61b130f08b None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.157571 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dffd32-06a0-4a7b-9a7e-5d61b130f08b None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.159623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 101/209] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:19.165735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5e0e9-d98a-4f9a-915d-a4ecba04801d None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.166284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5e0e9-d98a-4f9a-915d-a4ecba04801d None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.166284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5e0e9-d98a-4f9a-915d-a4ecba04801d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.186711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-74f5e0e9-d98a-4f9a-915d-a4ecba04801d None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:19.188339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-74f5e0e9-d98a-4f9a-915d-a4ecba04801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:19.259592 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 109/210] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:19.288214 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11429d-ebed-49f2-85e4-9fff8b3a229d None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.288513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11429d-ebed-49f2-85e4-9fff8b3a229d None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.288763 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11429d-ebed-49f2-85e4-9fff8b3a229d None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.290109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 102/211] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:19.295418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb49f1-9ec5-4f2a-b0e0-97d682a5d4fb None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.295949 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb49f1-9ec5-4f2a-b0e0-97d682a5d4fb None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.296414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb49f1-9ec5-4f2a-b0e0-97d682a5d4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.316351 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-99cb49f1-9ec5-4f2a-b0e0-97d682a5d4fb None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:19.317849 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-99cb49f1-9ec5-4f2a-b0e0-97d682a5d4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:19.380275 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:19.388093 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 110/212] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:19.542147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11429d-ebed-49f2-85e4-9fff8b3a229d None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:19.609711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc995b-e008-460a-8483-12deebce0083 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.610030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc995b-e008-460a-8483-12deebce0083 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.610262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc995b-e008-460a-8483-12deebce0083 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.611757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 111/213] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:19.616883 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd0325-9dba-4740-bfb6-c54f8bc98676 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.617162 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd0325-9dba-4740-bfb6-c54f8bc98676 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.617426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd0325-9dba-4740-bfb6-c54f8bc98676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.636411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-05cd0325-9dba-4740-bfb6-c54f8bc98676 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:19.637644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-05cd0325-9dba-4740-bfb6-c54f8bc98676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:19.659514 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11429d-ebed-49f2-85e4-9fff8b3a229d None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:19.663082 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:19.696914 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87142b-75a1-4c4f-b531-1bac026fd8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:19.697553 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87142b-75a1-4c4f-b531-1bac026fd8ba None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.697834 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87142b-75a1-4c4f-b531-1bac026fd8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.698105 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87142b-75a1-4c4f-b531-1bac026fd8ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.707349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 112/214] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:19.708209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd0325-9dba-4740-bfb6-c54f8bc98676 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:19.708824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87142b-75a1-4c4f-b531-1bac026fd8ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:19.713723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-ef87142b-75a1-4c4f-b531-1bac026fd8ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:19.713723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:19.713723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:19.713723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:19.713723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:19.713723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:19.713723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:19.713723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:19.713723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:19.713723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:19.713723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:19.713723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:19.713723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:19.713723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:19.713723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:19.713723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:19.714783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:19.714783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:19.714783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:19.714783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:19.714783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:19.714783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:19.714783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:19.714783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:19.714783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:19.714783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:19.714783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:19.714783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:19.714783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:19.714783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:19.714783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:19.714783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:19.715596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:19.715596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:19.715596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:19.715596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:19.715596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:19.715596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:19.715596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:19.715596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:19.715596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:19.715596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:19.715596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:19.715596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:19.715596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 103/215] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:19.716606 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fb05d-2d83-4a7a-8475-b79b5a6bbb99 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.716698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fb05d-2d83-4a7a-8475-b79b5a6bbb99 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.717000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fb05d-2d83-4a7a-8475-b79b5a6bbb99 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.718188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 104/216] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:19.723285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fb05d-2d83-4a7a-8475-b79b5a6bbb99 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:19.823914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd0325-9dba-4740-bfb6-c54f8bc98676 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:19.837336 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fb05d-2d83-4a7a-8475-b79b5a6bbb99 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:19.855302 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:19.864480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4449981-f66a-401f-9ac1-c5b0932ab5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:19.865122 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4449981-f66a-401f-9ac1-c5b0932ab5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.865844 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4449981-f66a-401f-9ac1-c5b0932ab5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.866118 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4449981-f66a-401f-9ac1-c5b0932ab5e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.867255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4449981-f66a-401f-9ac1-c5b0932ab5e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:19.868691 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4449981-f66a-401f-9ac1-c5b0932ab5e7 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:19.870448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 113/217] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:19.871560 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9ced3-9d39-4900-8d41-e7f3d3c5887e None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.871840 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9ced3-9d39-4900-8d41-e7f3d3c5887e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.872113 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9ced3-9d39-4900-8d41-e7f3d3c5887e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.880544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8cea3-fdea-44ff-9a3b-04e27607d4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:19.881443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8cea3-fdea-44ff-9a3b-04e27607d4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:19.881899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8cea3-fdea-44ff-9a3b-04e27607d4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:19.882365 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8cea3-fdea-44ff-9a3b-04e27607d4d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:19.883247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8cea3-fdea-44ff-9a3b-04e27607d4d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:19.883748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8cea3-fdea-44ff-9a3b-04e27607d4d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:19.885323 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8cea3-fdea-44ff-9a3b-04e27607d4d6 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:19.891378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-99a9ced3-9d39-4900-8d41-e7f3d3c5887e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:19.893135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-99a9ced3-9d39-4900-8d41-e7f3d3c5887e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:19.895551 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 105/218] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:19.897386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8cea3-fdea-44ff-9a3b-04e27607d4d6 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:19.915632 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:19.974763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 114/219] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:19.974763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9ced3-9d39-4900-8d41-e7f3d3c5887e None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:20.040622 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8cea3-fdea-44ff-9a3b-04e27607d4d6 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.078853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c476dec-0293-49ad-a134-138932f8da90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:20.079341 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c476dec-0293-49ad-a134-138932f8da90 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:20.079613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c476dec-0293-49ad-a134-138932f8da90 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:20.079919 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c476dec-0293-49ad-a134-138932f8da90 None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:20.081098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c476dec-0293-49ad-a134-138932f8da90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:20.082526 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c476dec-0293-49ad-a134-138932f8da90 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:20.098310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9ced3-9d39-4900-8d41-e7f3d3c5887e None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.107247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 106/220] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:19 2022] POST /identity/v3/users => generated 297 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:20.111771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c476dec-0293-49ad-a134-138932f8da90 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:20.143829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:20.160186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a20ab2-9de4-4283-b01f-707001da517c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:20.161810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a20ab2-9de4-4283-b01f-707001da517c None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:20.162296 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a20ab2-9de4-4283-b01f-707001da517c None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:20.163018 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a20ab2-9de4-4283-b01f-707001da517c None admin] PATH_INFO: `/v3/users/fe2049aff63c4d76a84c0a2cb92bb60b` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:20.164600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a20ab2-9de4-4283-b01f-707001da517c None admin] RBAC: Authorizing `identity:get_user(user_id=fe2049aff63c4d76a84c0a2cb92bb60b)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:20.166407 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a20ab2-9de4-4283-b01f-707001da517c None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:20.168999 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 115/221] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:19 2022] GET /identity/v3/users/fe2049aff63c4d76a84c0a2cb92bb60b => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:20.175410 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a20ab2-9de4-4283-b01f-707001da517c None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:20.217100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:20.268564 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c476dec-0293-49ad-a134-138932f8da90 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.309534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07240b0-364c-4ef3-8522-d85af993caef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:20.310185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07240b0-364c-4ef3-8522-d85af993caef None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:20.310459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07240b0-364c-4ef3-8522-d85af993caef None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:20.310742 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07240b0-364c-4ef3-8522-d85af993caef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:20.313661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a20ab2-9de4-4283-b01f-707001da517c None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.325374 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07240b0-364c-4ef3-8522-d85af993caef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:20.332139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-e07240b0-364c-4ef3-8522-d85af993caef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:20.332139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:20.332139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:20.332139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:20.332139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:20.332139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:20.332139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:20.332139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:20.332139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:20.332139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:20.332139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:20.332139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:20.332139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:20.332139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:20.332139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:20.332139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:20.332991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:20.332991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:20.332991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:20.332991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:20.332991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:20.332991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:20.332991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:20.332991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:20.332991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:20.332991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:20.332991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:20.332991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:20.332991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:20.332991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:20.332991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:20.332991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:20.333784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:20.333784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:20.333784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:20.333784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:20.333784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:20.333784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:20.333784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:20.333784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:20.333784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:20.333784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:20.333784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:20.333784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:20.333784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 107/222] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:20.334446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07240b0-364c-4ef3-8522-d85af993caef None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:20.367485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74518005-a923-43df-8ebb-595dca1a510e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:20.367946 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74518005-a923-43df-8ebb-595dca1a510e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:20.368267 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74518005-a923-43df-8ebb-595dca1a510e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:20.368532 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74518005-a923-43df-8ebb-595dca1a510e None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:20.369704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74518005-a923-43df-8ebb-595dca1a510e None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:20.372234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74518005-a923-43df-8ebb-595dca1a510e None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:20.374297 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 116/223] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:20 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:20.375254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74518005-a923-43df-8ebb-595dca1a510e None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:20.454673 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:20.474604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07240b0-364c-4ef3-8522-d85af993caef None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.509036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:20.517150 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74518005-a923-43df-8ebb-595dca1a510e None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.524163 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd7d13-1522-49ff-a5d2-c8fd21e8aea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:20.525118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd7d13-1522-49ff-a5d2-c8fd21e8aea6 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:20.525596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd7d13-1522-49ff-a5d2-c8fd21e8aea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:20.526084 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd7d13-1522-49ff-a5d2-c8fd21e8aea6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:20.537774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd7d13-1522-49ff-a5d2-c8fd21e8aea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:20.544408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-f8fd7d13-1522-49ff-a5d2-c8fd21e8aea6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:20.544408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:20.544408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:20.544408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:20.544408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:20.544408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:20.544408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:20.544408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:20.544408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:20.544408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:20.544408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:20.544408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:20.544408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:20.544408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:20.544408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:20.544408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:20.545312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:20.545312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:20.545312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:20.545312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:20.545312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:20.545312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:20.545312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:20.545312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:20.545312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:20.545312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:20.545312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:20.545312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:20.545312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:20.545312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:20.545312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:20.545312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:20.546122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:20.546122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:20.546122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:20.546122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:20.546122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:20.546122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:20.546122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:20.546122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:20.546122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:20.546122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:20.546122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:20.546122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:20.548750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 108/224] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:44:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:20.550263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd7d13-1522-49ff-a5d2-c8fd21e8aea6 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:20.554688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecb5ab-2383-4be8-b8c7-19ebbc5df42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:20.555347 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecb5ab-2383-4be8-b8c7-19ebbc5df42c None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:20.555721 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecb5ab-2383-4be8-b8c7-19ebbc5df42c None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:20.556001 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecb5ab-2383-4be8-b8c7-19ebbc5df42c None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:20.556679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ecb5ab-2383-4be8-b8c7-19ebbc5df42c None admin] RBAC: Adding query filter params (name=member) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:20.556986 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ecb5ab-2383-4be8-b8c7-19ebbc5df42c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:20.558371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ecb5ab-2383-4be8-b8c7-19ebbc5df42c None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:20.567803 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 117/225] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:20.568675 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecb5ab-2383-4be8-b8c7-19ebbc5df42c None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:20.675680 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd7d13-1522-49ff-a5d2-c8fd21e8aea6 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.683005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecb5ab-2383-4be8-b8c7-19ebbc5df42c None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.707595 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:20.714327 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791fb6f-4fb4-4085-9cb8-4ac318d9823e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:20.715020 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791fb6f-4fb4-4085-9cb8-4ac318d9823e None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:20.715405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791fb6f-4fb4-4085-9cb8-4ac318d9823e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:20.715679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791fb6f-4fb4-4085-9cb8-4ac318d9823e None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d/users/fe2049aff63c4d76a84c0a2cb92bb60b/roles/1b632105fdba460b809a2ecd1d186409` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:20.718505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791fb6f-4fb4-4085-9cb8-4ac318d9823e None admin] RBAC: Authorizing `identity:create_grant(project_id=45e79ea1241346c58bf04c1494b3dc7d, user_id=fe2049aff63c4d76a84c0a2cb92bb60b, role_id=1b632105fdba460b809a2ecd1d186409)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:20.722112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791fb6f-4fb4-4085-9cb8-4ac318d9823e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:20.724494 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d7283-8b5c-4349-bd1b-7592d90c2004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:20.725452 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d7283-8b5c-4349-bd1b-7592d90c2004 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:20.726086 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d7283-8b5c-4349-bd1b-7592d90c2004 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:20.726635 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d7283-8b5c-4349-bd1b-7592d90c2004 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:20.727524 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d7283-8b5c-4349-bd1b-7592d90c2004 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:20.727917 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d7283-8b5c-4349-bd1b-7592d90c2004 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:20.729541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d7283-8b5c-4349-bd1b-7592d90c2004 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:20.737349 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 109/226] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:20 2022] PUT /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d/users/fe2049aff63c4d76a84c0a2cb92bb60b/roles/1b632105fdba460b809a2ecd1d186409 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:20.742876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 118/227] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:20.742876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d7283-8b5c-4349-bd1b-7592d90c2004 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:20.752924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791fb6f-4fb4-4085-9cb8-4ac318d9823e None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:20.852566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:20.950930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d7283-8b5c-4349-bd1b-7592d90c2004 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.958134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791fb6f-4fb4-4085-9cb8-4ac318d9823e None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:20.992897 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f00c91-5b8f-40a6-8a98-0e8e5aad093d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:20.993538 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f00c91-5b8f-40a6-8a98-0e8e5aad093d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:20.993819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f00c91-5b8f-40a6-8a98-0e8e5aad093d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:20.994099 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f00c91-5b8f-40a6-8a98-0e8e5aad093d None admin] PATH_INFO: `/v3/users/b054fc3565b54141a10d7a43e5d21904` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:21.009663 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c88e3-df5f-48bf-ad57-2e3a01247f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:21.011439 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c88e3-df5f-48bf-ad57-2e3a01247f6c None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:21.012027 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c88e3-df5f-48bf-ad57-2e3a01247f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:21.012582 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c88e3-df5f-48bf-ad57-2e3a01247f6c None admin] PATH_INFO: `/v3/users/fe2049aff63c4d76a84c0a2cb92bb60b` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:21.012883 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f00c91-5b8f-40a6-8a98-0e8e5aad093d None admin] RBAC: Authorizing `identity:get_user(user_id=b054fc3565b54141a10d7a43e5d21904)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:21.014185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c88e3-df5f-48bf-ad57-2e3a01247f6c None admin] RBAC: Authorizing `identity:get_user(user_id=fe2049aff63c4d76a84c0a2cb92bb60b)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:21.014380 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f00c91-5b8f-40a6-8a98-0e8e5aad093d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:21.016283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 119/228] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:20 2022] GET /identity/v3/users/b054fc3565b54141a10d7a43e5d21904 => generated 297 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:21.017500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c88e3-df5f-48bf-ad57-2e3a01247f6c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:21.019736 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 110/229] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:20 2022] GET /identity/v3/users/fe2049aff63c4d76a84c0a2cb92bb60b => generated 283 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:21.025249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c88e3-df5f-48bf-ad57-2e3a01247f6c None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:21.025387 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f00c91-5b8f-40a6-8a98-0e8e5aad093d None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:21.073611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:21.135836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c88e3-df5f-48bf-ad57-2e3a01247f6c None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:21.136960 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f00c91-5b8f-40a6-8a98-0e8e5aad093d None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:21.159157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:21.172794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95a8b4-f669-4072-aad7-abebda4efb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:21.173846 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba881f-762e-430f-aa36-e0e0795ddf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:21.174475 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba881f-762e-430f-aa36-e0e0795ddf0a None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:21.174897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95a8b4-f669-4072-aad7-abebda4efb17 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:21.175023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba881f-762e-430f-aa36-e0e0795ddf0a None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:21.175187 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba881f-762e-430f-aa36-e0e0795ddf0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:21.175650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95a8b4-f669-4072-aad7-abebda4efb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:21.176190 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95a8b4-f669-4072-aad7-abebda4efb17 None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:21.177556 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95a8b4-f669-4072-aad7-abebda4efb17 None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:21.179265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95a8b4-f669-4072-aad7-abebda4efb17 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:21.181163 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 111/230] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:21 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:21.187037 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95a8b4-f669-4072-aad7-abebda4efb17 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:21.273492 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba881f-762e-430f-aa36-e0e0795ddf0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:21.274909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba881f-762e-430f-aa36-e0e0795ddf0a None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:21.291713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95a8b4-f669-4072-aad7-abebda4efb17 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:21.320917 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:21.341138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca9f5f-ed38-4a36-af70-05c9fefa6469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:21.342212 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca9f5f-ed38-4a36-af70-05c9fefa6469 None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:21.342696 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca9f5f-ed38-4a36-af70-05c9fefa6469 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:21.343238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca9f5f-ed38-4a36-af70-05c9fefa6469 None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d/users/fe2049aff63c4d76a84c0a2cb92bb60b/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:21.345492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ca9f5f-ed38-4a36-af70-05c9fefa6469 None admin] RBAC: Authorizing `identity:create_grant(project_id=45e79ea1241346c58bf04c1494b3dc7d, user_id=fe2049aff63c4d76a84c0a2cb92bb60b, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:21.347740 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ca9f5f-ed38-4a36-af70-05c9fefa6469 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:21.366047 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 112/231] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:21 2022] PUT /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d/users/fe2049aff63c4d76a84c0a2cb92bb60b/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:44:21.389025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 120/232] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:21.395881 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca9f5f-ed38-4a36-af70-05c9fefa6469 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:21.476507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:21.562541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca9f5f-ed38-4a36-af70-05c9fefa6469 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:21.611647 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5b88f-d57b-40a4-83e9-e7e9f6bc23e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:21.612322 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5b88f-d57b-40a4-83e9-e7e9f6bc23e9 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:21.612604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5b88f-d57b-40a4-83e9-e7e9f6bc23e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:21.612882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5b88f-d57b-40a4-83e9-e7e9f6bc23e9 None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:21.614149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5b88f-d57b-40a4-83e9-e7e9f6bc23e9 None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:21.616120 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5b88f-d57b-40a4-83e9-e7e9f6bc23e9 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:21.618977 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 113/233] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:21 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:21.625319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba881f-762e-430f-aa36-e0e0795ddf0a None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:21.765962 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba881f-762e-430f-aa36-e0e0795ddf0a None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:21.809827 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53935ff-e463-4c5c-86c2-78c3b254bc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:21.810434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53935ff-e463-4c5c-86c2-78c3b254bc79 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:21.810704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53935ff-e463-4c5c-86c2-78c3b254bc79 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:21.811010 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53935ff-e463-4c5c-86c2-78c3b254bc79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:21.811733 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53935ff-e463-4c5c-86c2-78c3b254bc79 None admin] RBAC: Adding query filter params (role.id=7d3cf58c1ad2407c9f8c60ba99b2006d, scope.project.id=45e79ea1241346c58bf04c1494b3dc7d, user.id=b054fc3565b54141a10d7a43e5d21904) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:21.812036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53935ff-e463-4c5c-86c2-78c3b254bc79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:21.813514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53935ff-e463-4c5c-86c2-78c3b254bc79 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:21.820258 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 121/234] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:21 2022] GET /identity/v3/role_assignments?user.id=b054fc3565b54141a10d7a43e5d21904&scope.project.id=45e79ea1241346c58bf04c1494b3dc7d&role.id=7d3cf58c1ad2407c9f8c60ba99b2006d => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:23.035461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218eebe-92ac-43f6-9c4d-fb5a16d20f6c None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.036236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218eebe-92ac-43f6-9c4d-fb5a16d20f6c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.036236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218eebe-92ac-43f6-9c4d-fb5a16d20f6c None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.038492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 114/235] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:23.056744 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6e0ac-d735-4e9b-9196-007f13503564 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.057498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6e0ac-d735-4e9b-9196-007f13503564 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.057498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6e0ac-d735-4e9b-9196-007f13503564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.078067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:23.084140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-d4b6e0ac-d735-4e9b-9196-007f13503564 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:23.085321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d4b6e0ac-d735-4e9b-9196-007f13503564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:23.114506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f2c85-0aa9-49fa-a5fb-88229b27dfdd None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.115449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f2c85-0aa9-49fa-a5fb-88229b27dfdd None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.116200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f2c85-0aa9-49fa-a5fb-88229b27dfdd None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.119222 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 115/236] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:23.129106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd302f2-2877-4c45-bbef-0a77be8065bc None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.129963 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd302f2-2877-4c45-bbef-0a77be8065bc None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.130780 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd302f2-2877-4c45-bbef-0a77be8065bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.161570 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-fcd302f2-2877-4c45-bbef-0a77be8065bc None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:23.164325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-fcd302f2-2877-4c45-bbef-0a77be8065bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:23.184373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 122/237] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:23.286300 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 116/238] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:23.489564 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6e0ac-d735-4e9b-9196-007f13503564 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:23.613589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6e0ac-d735-4e9b-9196-007f13503564 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:23.663462 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31c09c-df29-45f4-b583-98a135ec78db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:23.664116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31c09c-df29-45f4-b583-98a135ec78db None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.664391 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31c09c-df29-45f4-b583-98a135ec78db None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.664663 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31c09c-df29-45f4-b583-98a135ec78db None admin] PATH_INFO: `/v3/domains/default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.665819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31c09c-df29-45f4-b583-98a135ec78db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:23.667285 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31c09c-df29-45f4-b583-98a135ec78db None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:23.668856 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 123/239] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:23.675267 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd302f2-2877-4c45-bbef-0a77be8065bc None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:23.723580 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31c09c-df29-45f4-b583-98a135ec78db None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:23.735906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:23.779294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:23.866659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd302f2-2877-4c45-bbef-0a77be8065bc None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:23.895447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31c09c-df29-45f4-b583-98a135ec78db None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:23.913707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47aefa-ec1a-4776-a666-46480533f51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:23.914841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47aefa-ec1a-4776-a666-46480533f51b None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.915388 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47aefa-ec1a-4776-a666-46480533f51b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.915747 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47aefa-ec1a-4776-a666-46480533f51b None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.916474 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47aefa-ec1a-4776-a666-46480533f51b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:23.918553 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47aefa-ec1a-4776-a666-46480533f51b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:23.933816 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54199b58-cd4e-4a54-8a40-9f5afe55e590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:23.934803 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54199b58-cd4e-4a54-8a40-9f5afe55e590 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.935346 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54199b58-cd4e-4a54-8a40-9f5afe55e590 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.935828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54199b58-cd4e-4a54-8a40-9f5afe55e590 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.948331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 117/240] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:23 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:23.949770 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f79685-4705-4843-ad36-2a86a2711e61 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.950308 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f79685-4705-4843-ad36-2a86a2711e61 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.951012 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f79685-4705-4843-ad36-2a86a2711e61 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.952551 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 118/241] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:23.963445 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54199b58-cd4e-4a54-8a40-9f5afe55e590 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:23.969188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-54199b58-cd4e-4a54-8a40-9f5afe55e590 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:23.969188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:23.969188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:23.969188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:23.969188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:23.969188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:23.969188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:23.969188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:23.969188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:23.969188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:23.969188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:23.969188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:23.969188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:23.969188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:23.969188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:23.969188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:23.970334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:23.970334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:23.970334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:23.970334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:23.970334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:23.970334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:23.970334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:23.970334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:23.970334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:23.970334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:23.970334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:23.970334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:23.970334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:23.970334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:23.970334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:23.970334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:23.973720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:23.973720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:23.973720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:23.973720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:23.973720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:23.973720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:23.973720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:23.973720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:23.973720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:23.973720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:23.973720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:23.973720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:23.976143 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d77d2-8b36-4455-8f49-1336fefa4fb9 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:23.976611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d77d2-8b36-4455-8f49-1336fefa4fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:23.977114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d77d2-8b36-4455-8f49-1336fefa4fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:23.985450 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 124/242] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:23.991146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54199b58-cd4e-4a54-8a40-9f5afe55e590 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:24.019443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-668d77d2-8b36-4455-8f49-1336fefa4fb9 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:24.022893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-668d77d2-8b36-4455-8f49-1336fefa4fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:24.136961 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:24.178678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 119/243] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:24.194272 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54199b58-cd4e-4a54-8a40-9f5afe55e590 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:24.232564 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2915f1-2374-4b72-b164-ff8cd09da3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:24.233448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2915f1-2374-4b72-b164-ff8cd09da3bd None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:24.233866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2915f1-2374-4b72-b164-ff8cd09da3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:24.234273 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2915f1-2374-4b72-b164-ff8cd09da3bd None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:24.235103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2915f1-2374-4b72-b164-ff8cd09da3bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:24.235541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2915f1-2374-4b72-b164-ff8cd09da3bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:24.237185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2915f1-2374-4b72-b164-ff8cd09da3bd None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:24.246449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 125/244] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:24.258067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d77d2-8b36-4455-8f49-1336fefa4fb9 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:24.290667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:24.410211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d77d2-8b36-4455-8f49-1336fefa4fb9 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:24.452095 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67ce19-95ae-4de4-b251-4d309a8d1816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:24.452877 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67ce19-95ae-4de4-b251-4d309a8d1816 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:24.453192 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67ce19-95ae-4de4-b251-4d309a8d1816 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:24.453459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d67ce19-95ae-4de4-b251-4d309a8d1816 None admin] PATH_INFO: `/v3/users/d8e797526d17487da511844ebbdb19db` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:24.469933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67ce19-95ae-4de4-b251-4d309a8d1816 None admin] RBAC: Authorizing `identity:get_user(user_id=d8e797526d17487da511844ebbdb19db)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:24.471715 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d67ce19-95ae-4de4-b251-4d309a8d1816 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:24.473454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 120/245] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:24 2022] GET /identity/v3/users/d8e797526d17487da511844ebbdb19db => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:24.479139 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2915f1-2374-4b72-b164-ff8cd09da3bd None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:24.495352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67ce19-95ae-4de4-b251-4d309a8d1816 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:24.595615 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:24.603552 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2915f1-2374-4b72-b164-ff8cd09da3bd None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:24.613510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d67ce19-95ae-4de4-b251-4d309a8d1816 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:24.638790 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:24.658384 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a156d-a5b4-432f-aaa5-8b5ac607f190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:24.659521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a156d-a5b4-432f-aaa5-8b5ac607f190 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:24.660017 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a156d-a5b4-432f-aaa5-8b5ac607f190 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:24.660531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a156d-a5b4-432f-aaa5-8b5ac607f190 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:24.665786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5056220-1be6-4e75-976c-19fcf2493b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:24.666762 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5056220-1be6-4e75-976c-19fcf2493b9c None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:24.667241 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5056220-1be6-4e75-976c-19fcf2493b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:24.667729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5056220-1be6-4e75-976c-19fcf2493b9c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:24.683387 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5056220-1be6-4e75-976c-19fcf2493b9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:24.690635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-e5056220-1be6-4e75-976c-19fcf2493b9c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:24.690635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:24.690635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:24.690635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:24.690635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:24.690635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:24.690635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:24.690635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:24.690635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:24.690635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:24.690635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:24.690635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:24.690635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:24.690635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:24.690635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:24.690635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:24.691563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:24.691563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:24.691563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:24.691563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:24.691563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:24.691563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:24.691563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:24.691563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:24.691563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:24.691563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:24.691563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:24.691563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:24.691563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:24.691563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:24.691563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:24.691563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:24.692366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:24.692366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:24.692366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:24.692366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:24.692366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:24.692366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:24.692366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:24.692366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:24.692366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:24.692366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:24.692366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:24.692366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:24.694994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 121/246] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:24.696366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40618c62-ae97-4f80-8689-cd83b1922aab None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:24.696797 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40618c62-ae97-4f80-8689-cd83b1922aab None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:24.697193 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40618c62-ae97-4f80-8689-cd83b1922aab None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:24.698873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 122/247] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:24.703559 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40618c62-ae97-4f80-8689-cd83b1922aab None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:24.764839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a156d-a5b4-432f-aaa5-8b5ac607f190 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:24.766465 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a156d-a5b4-432f-aaa5-8b5ac607f190 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:24.834766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40618c62-ae97-4f80-8689-cd83b1922aab None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:24.871739 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a16c6e-715e-4d0e-91aa-be0d4794f2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:24.872354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a16c6e-715e-4d0e-91aa-be0d4794f2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:24.872630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a16c6e-715e-4d0e-91aa-be0d4794f2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:24.872903 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a16c6e-715e-4d0e-91aa-be0d4794f2b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:24.873567 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a16c6e-715e-4d0e-91aa-be0d4794f2b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:24.873868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a16c6e-715e-4d0e-91aa-be0d4794f2b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:24.875305 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a16c6e-715e-4d0e-91aa-be0d4794f2b7 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:24.884822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 123/248] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:24.885894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cce25e-83d3-48a5-b2ef-4351e7f77713 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:24.886176 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cce25e-83d3-48a5-b2ef-4351e7f77713 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:24.886504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cce25e-83d3-48a5-b2ef-4351e7f77713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:24.900721 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:24.906436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-e6cce25e-83d3-48a5-b2ef-4351e7f77713 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:24.906846 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 126/249] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:24.907836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e6cce25e-83d3-48a5-b2ef-4351e7f77713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:24.908293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ba9eb-8867-46c7-9893-bbd53e03de67 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:24.908587 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ba9eb-8867-46c7-9893-bbd53e03de67 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:24.908865 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ba9eb-8867-46c7-9893-bbd53e03de67 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:24.910589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 127/250] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:24.912627 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ba9eb-8867-46c7-9893-bbd53e03de67 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:24.949322 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:24.989544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 124/251] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:24.990540 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cce25e-83d3-48a5-b2ef-4351e7f77713 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:25.058275 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ba9eb-8867-46c7-9893-bbd53e03de67 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:25.114530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080df41-641c-4a20-bad9-fc27338baa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:25.115200 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080df41-641c-4a20-bad9-fc27338baa35 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:25.115473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080df41-641c-4a20-bad9-fc27338baa35 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:25.115742 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080df41-641c-4a20-bad9-fc27338baa35 None admin] PATH_INFO: `/v3/users/fe2049aff63c4d76a84c0a2cb92bb60b` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:25.117686 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080df41-641c-4a20-bad9-fc27338baa35 None admin] RBAC: Authorizing `identity:get_user(user_id=fe2049aff63c4d76a84c0a2cb92bb60b)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:25.119149 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080df41-641c-4a20-bad9-fc27338baa35 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:25.120728 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 128/252] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:24 2022] GET /identity/v3/users/fe2049aff63c4d76a84c0a2cb92bb60b => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:25.123058 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16a784-337e-46e1-a6d9-88e83e139af6 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:25.123333 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16a784-337e-46e1-a6d9-88e83e139af6 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:25.123598 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16a784-337e-46e1-a6d9-88e83e139af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:25.156066 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cce25e-83d3-48a5-b2ef-4351e7f77713 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:25.169397 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-8f16a784-337e-46e1-a6d9-88e83e139af6 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:25.170514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-8f16a784-337e-46e1-a6d9-88e83e139af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:25.228607 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459fd4f-20dd-4811-8038-21197685ce19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:25.230034 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459fd4f-20dd-4811-8038-21197685ce19 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:25.230533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459fd4f-20dd-4811-8038-21197685ce19 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:25.231020 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459fd4f-20dd-4811-8038-21197685ce19 None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:25.232366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459fd4f-20dd-4811-8038-21197685ce19 None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:25.234019 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459fd4f-20dd-4811-8038-21197685ce19 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:25.235923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 125/253] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:24 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:25.237579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459fd4f-20dd-4811-8038-21197685ce19 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:25.247048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:25.285175 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 129/254] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:25.286137 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16a784-337e-46e1-a6d9-88e83e139af6 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:25.318246 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:25.400149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459fd4f-20dd-4811-8038-21197685ce19 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:25.403952 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16a784-337e-46e1-a6d9-88e83e139af6 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:25.446512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08d23b-c589-479d-97b9-34fb6ded9068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:25.447417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08d23b-c589-479d-97b9-34fb6ded9068 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:25.447892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08d23b-c589-479d-97b9-34fb6ded9068 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:25.448344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08d23b-c589-479d-97b9-34fb6ded9068 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:25.449346 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08d23b-c589-479d-97b9-34fb6ded9068 None admin] RBAC: Adding query filter params (role.id=77e7991dcd664ba086355ef20ff81541, scope.project.id=45e79ea1241346c58bf04c1494b3dc7d, user.id=d8e797526d17487da511844ebbdb19db) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:25.449829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08d23b-c589-479d-97b9-34fb6ded9068 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:25.451618 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08d23b-c589-479d-97b9-34fb6ded9068 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:25.458456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 130/255] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:25 2022] GET /identity/v3/role_assignments?user.id=d8e797526d17487da511844ebbdb19db&scope.project.id=45e79ea1241346c58bf04c1494b3dc7d&role.id=77e7991dcd664ba086355ef20ff81541 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:25.463886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08d23b-c589-479d-97b9-34fb6ded9068 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:25.498914 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5b038-acdd-47fd-a559-32e5ed7b0995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:25.498914 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5b038-acdd-47fd-a559-32e5ed7b0995 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:25.498914 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5b038-acdd-47fd-a559-32e5ed7b0995 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:25.498914 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5b038-acdd-47fd-a559-32e5ed7b0995 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:25.642706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:25.649062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f5b038-acdd-47fd-a559-32e5ed7b0995 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:25.650593 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f5b038-acdd-47fd-a559-32e5ed7b0995 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:25.658755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:25.674489 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08d23b-c589-479d-97b9-34fb6ded9068 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:25.751538 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb7370-f5ec-4692-91e1-7ef3199e5d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:25.752685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb7370-f5ec-4692-91e1-7ef3199e5d40 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:25.752970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb7370-f5ec-4692-91e1-7ef3199e5d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:25.753241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb7370-f5ec-4692-91e1-7ef3199e5d40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:25.770359 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb7370-f5ec-4692-91e1-7ef3199e5d40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:25.777866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-31bb7370-f5ec-4692-91e1-7ef3199e5d40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:25.777866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:25.777866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:25.777866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:25.777866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:25.777866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:25.777866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:25.777866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:25.777866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:25.777866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:25.777866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:25.777866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:25.777866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:25.777866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:25.777866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:25.777866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:25.778854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:25.778854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:25.778854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:25.778854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:25.778854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:25.778854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:25.778854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:25.778854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:25.778854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:25.778854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:25.778854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:25.778854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:25.778854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:25.778854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:25.778854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:25.778854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:25.779683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:25.779683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:25.779683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:25.779683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:25.779683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:25.779683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:25.779683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:25.779683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:25.779683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:25.779683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:25.779683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:25.779683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:25.779683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 131/256] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:44:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:25.780338 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb7370-f5ec-4692-91e1-7ef3199e5d40 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:25.857760 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 126/257] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:25.858621 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5b038-acdd-47fd-a559-32e5ed7b0995 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:25.961516 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb7370-f5ec-4692-91e1-7ef3199e5d40 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:26.009093 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41f87f-0586-4179-8f4c-73d828a997ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:26.010055 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41f87f-0586-4179-8f4c-73d828a997ff None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:26.010527 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41f87f-0586-4179-8f4c-73d828a997ff None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:26.011133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41f87f-0586-4179-8f4c-73d828a997ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:26.024612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db41f87f-0586-4179-8f4c-73d828a997ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:26.030064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-db41f87f-0586-4179-8f4c-73d828a997ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:26.030064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:26.030064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:26.030064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:26.030064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:26.030064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:26.030064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:26.030064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:26.030064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:26.030064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:26.030064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:26.030064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:26.030064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:26.030064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:26.030064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:26.030064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:26.030985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:26.030985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:26.030985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:26.030985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:26.030985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:26.030985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:26.030985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:26.030985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:26.030985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:26.030985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:26.030985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:26.030985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:26.030985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:26.030985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:26.030985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:26.030985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:26.031781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:26.031781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:26.031781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:26.031781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:26.031781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:26.031781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:26.031781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:26.031781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:26.031781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:26.031781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:26.031781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:26.031781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:26.034532 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 132/258] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:26.035686 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41f87f-0586-4179-8f4c-73d828a997ff None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:26.052354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:26.084009 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5b038-acdd-47fd-a559-32e5ed7b0995 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:26.088440 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:26.148577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41f87f-0586-4179-8f4c-73d828a997ff None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:26.152123 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84245c85-ad4e-482a-bee7-b0a4c94876b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:26.152593 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84245c85-ad4e-482a-bee7-b0a4c94876b2 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:26.152934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84245c85-ad4e-482a-bee7-b0a4c94876b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:26.153215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84245c85-ad4e-482a-bee7-b0a4c94876b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:26.153886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84245c85-ad4e-482a-bee7-b0a4c94876b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:26.154190 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84245c85-ad4e-482a-bee7-b0a4c94876b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:26.155610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84245c85-ad4e-482a-bee7-b0a4c94876b2 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:26.174695 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 127/259] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:26.175653 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84245c85-ad4e-482a-bee7-b0a4c94876b2 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:26.186483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c55092-fc18-480c-bcfa-99dcced44cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:26.187814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c55092-fc18-480c-bcfa-99dcced44cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:26.188304 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c55092-fc18-480c-bcfa-99dcced44cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:26.188778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c55092-fc18-480c-bcfa-99dcced44cd7 None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:26.190890 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c55092-fc18-480c-bcfa-99dcced44cd7 None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:26.192553 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c55092-fc18-480c-bcfa-99dcced44cd7 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:26.194387 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 133/260] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:26 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:26.195486 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c55092-fc18-480c-bcfa-99dcced44cd7 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:26.277976 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:26.303177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c55092-fc18-480c-bcfa-99dcced44cd7 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:26.342083 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88786c6c-4bd3-4fa1-a6b8-907226fcf21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:26.343077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88786c6c-4bd3-4fa1-a6b8-907226fcf21c None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:26.343592 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88786c6c-4bd3-4fa1-a6b8-907226fcf21c None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:26.344196 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88786c6c-4bd3-4fa1-a6b8-907226fcf21c None admin] PATH_INFO: `/v3/users/fe2049aff63c4d76a84c0a2cb92bb60b` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:26.346412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88786c6c-4bd3-4fa1-a6b8-907226fcf21c None admin] RBAC: Authorizing `identity:get_user(user_id=fe2049aff63c4d76a84c0a2cb92bb60b)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:26.348159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88786c6c-4bd3-4fa1-a6b8-907226fcf21c None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:26.350375 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 134/261] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:26 2022] GET /identity/v3/users/fe2049aff63c4d76a84c0a2cb92bb60b => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:26.351710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88786c6c-4bd3-4fa1-a6b8-907226fcf21c None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:26.425380 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84245c85-ad4e-482a-bee7-b0a4c94876b2 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:26.468890 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88786c6c-4bd3-4fa1-a6b8-907226fcf21c None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:26.506118 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61049ead-8d81-4699-9348-8b7d0564e5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:26.507189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61049ead-8d81-4699-9348-8b7d0564e5bb None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:26.507697 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61049ead-8d81-4699-9348-8b7d0564e5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:26.508160 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61049ead-8d81-4699-9348-8b7d0564e5bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:26.509116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61049ead-8d81-4699-9348-8b7d0564e5bb None admin] RBAC: Adding query filter params (role.id=1b632105fdba460b809a2ecd1d186409, scope.project.id=45e79ea1241346c58bf04c1494b3dc7d, user.id=fe2049aff63c4d76a84c0a2cb92bb60b) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:26.509663 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61049ead-8d81-4699-9348-8b7d0564e5bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:26.511399 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61049ead-8d81-4699-9348-8b7d0564e5bb None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:26.518107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 135/262] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:26 2022] GET /identity/v3/role_assignments?user.id=fe2049aff63c4d76a84c0a2cb92bb60b&scope.project.id=45e79ea1241346c58bf04c1494b3dc7d&role.id=1b632105fdba460b809a2ecd1d186409 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:26.519354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61049ead-8d81-4699-9348-8b7d0564e5bb None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:26.519808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606df19-e598-4fb8-9752-2d98e90dc99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:26.520442 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606df19-e598-4fb8-9752-2d98e90dc99d None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:26.520739 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606df19-e598-4fb8-9752-2d98e90dc99d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:26.520988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606df19-e598-4fb8-9752-2d98e90dc99d None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:26.521666 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606df19-e598-4fb8-9752-2d98e90dc99d None admin] RBAC: Adding query filter params (name=member) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:26.521971 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606df19-e598-4fb8-9752-2d98e90dc99d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:26.523559 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606df19-e598-4fb8-9752-2d98e90dc99d None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:26.542024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 128/263] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:26.542431 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:26.547607 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606df19-e598-4fb8-9752-2d98e90dc99d None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:26.612501 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:26.639442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61049ead-8d81-4699-9348-8b7d0564e5bb None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:26.676631 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71578f4e-0a50-4ae8-a6e6-9b935fce3e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:26.677632 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71578f4e-0a50-4ae8-a6e6-9b935fce3e02 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:26.678101 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71578f4e-0a50-4ae8-a6e6-9b935fce3e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:26.678558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71578f4e-0a50-4ae8-a6e6-9b935fce3e02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:26.760367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71578f4e-0a50-4ae8-a6e6-9b935fce3e02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:26.762202 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71578f4e-0a50-4ae8-a6e6-9b935fce3e02 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:26.783826 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:26.868755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606df19-e598-4fb8-9752-2d98e90dc99d None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:26.886847 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 136/264] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:26.894106 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71578f4e-0a50-4ae8-a6e6-9b935fce3e02 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:26.944552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee9552-b5f0-4c5a-b44c-2cabd0309a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:26.945212 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee9552-b5f0-4c5a-b44c-2cabd0309a55 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:26.945482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee9552-b5f0-4c5a-b44c-2cabd0309a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:26.945754 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee9552-b5f0-4c5a-b44c-2cabd0309a55 None admin] PATH_INFO: `/v3/users/b054fc3565b54141a10d7a43e5d21904` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:26.947936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee9552-b5f0-4c5a-b44c-2cabd0309a55 None admin] RBAC: Authorizing `identity:get_user(user_id=b054fc3565b54141a10d7a43e5d21904)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:26.949456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee9552-b5f0-4c5a-b44c-2cabd0309a55 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:26.951219 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 129/265] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:26 2022] GET /identity/v3/users/b054fc3565b54141a10d7a43e5d21904 => generated 297 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:26.957386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee9552-b5f0-4c5a-b44c-2cabd0309a55 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:27.083785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71578f4e-0a50-4ae8-a6e6-9b935fce3e02 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:27.092122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:27.096949 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:27.101611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee9552-b5f0-4c5a-b44c-2cabd0309a55 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:27.128046 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d7573-fdbc-4d26-8411-3831e1ed9c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:27.128608 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d7573-fdbc-4d26-8411-3831e1ed9c42 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:27.128888 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d7573-fdbc-4d26-8411-3831e1ed9c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:27.129155 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d7573-fdbc-4d26-8411-3831e1ed9c42 None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:27.130378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d7573-fdbc-4d26-8411-3831e1ed9c42 None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:27.132793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d7573-fdbc-4d26-8411-3831e1ed9c42 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:27.139085 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 137/266] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:26 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:27.140987 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914b708-fde1-4225-8a2c-da04ce128aef None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:27.141284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914b708-fde1-4225-8a2c-da04ce128aef None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:27.141560 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914b708-fde1-4225-8a2c-da04ce128aef None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:27.143693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 138/267] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:27.145384 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914b708-fde1-4225-8a2c-da04ce128aef None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:27.162758 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47796a0-7697-4bfb-9ffd-add68e912d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:27.162758 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47796a0-7697-4bfb-9ffd-add68e912d02 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:27.162758 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47796a0-7697-4bfb-9ffd-add68e912d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:27.162758 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47796a0-7697-4bfb-9ffd-add68e912d02 None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:27.163145 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47796a0-7697-4bfb-9ffd-add68e912d02 None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:27.165139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47796a0-7697-4bfb-9ffd-add68e912d02 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:27.167693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 130/268] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:26 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:27.169191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f9c93-6b58-46ba-8c19-5eed8515216d None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:27.169468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f9c93-6b58-46ba-8c19-5eed8515216d None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:27.169736 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f9c93-6b58-46ba-8c19-5eed8515216d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:27.195916 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-959f9c93-6b58-46ba-8c19-5eed8515216d None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:27.197122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-959f9c93-6b58-46ba-8c19-5eed8515216d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:27.284759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914b708-fde1-4225-8a2c-da04ce128aef None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:27.294249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 131/269] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:27.294930 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f9c93-6b58-46ba-8c19-5eed8515216d None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:27.359941 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711cad3-9255-4ca4-a4db-3b75a6844111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:27.360561 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711cad3-9255-4ca4-a4db-3b75a6844111 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:27.360839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711cad3-9255-4ca4-a4db-3b75a6844111 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:27.361111 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711cad3-9255-4ca4-a4db-3b75a6844111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:27.361904 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711cad3-9255-4ca4-a4db-3b75a6844111 None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=45e79ea1241346c58bf04c1494b3dc7d, user.id=fe2049aff63c4d76a84c0a2cb92bb60b) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:27.362294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711cad3-9255-4ca4-a4db-3b75a6844111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:27.363867 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711cad3-9255-4ca4-a4db-3b75a6844111 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:27.372976 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 139/270] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:27 2022] GET /identity/v3/role_assignments?user.id=fe2049aff63c4d76a84c0a2cb92bb60b&scope.project.id=45e79ea1241346c58bf04c1494b3dc7d&role.id=32c8a81e339a462e973e76bb528ef35a => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:27.374185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d108362-ed7e-4fad-bf61-737684cd27a5 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:27.374564 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d108362-ed7e-4fad-bf61-737684cd27a5 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:27.374879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d108362-ed7e-4fad-bf61-737684cd27a5 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:27.376076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 140/271] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:27.389106 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9da67e-3149-49d6-851f-102027156a0a None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:27.389299 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9da67e-3149-49d6-851f-102027156a0a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:27.389565 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9da67e-3149-49d6-851f-102027156a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:27.427074 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-dd9da67e-3149-49d6-851f-102027156a0a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:27.428334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-dd9da67e-3149-49d6-851f-102027156a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:27.510023 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:27.514851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:27.528106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f9c93-6b58-46ba-8c19-5eed8515216d None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:27.582294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 141/272] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:27.590593 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654c9a0-2689-4c95-b280-a0628fac97ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:27.591387 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654c9a0-2689-4c95-b280-a0628fac97ed None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:27.591734 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654c9a0-2689-4c95-b280-a0628fac97ed None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:27.592077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654c9a0-2689-4c95-b280-a0628fac97ed None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d/users/b054fc3565b54141a10d7a43e5d21904/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:27.594071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654c9a0-2689-4c95-b280-a0628fac97ed None admin] RBAC: Authorizing `identity:create_grant(project_id=45e79ea1241346c58bf04c1494b3dc7d, user_id=b054fc3565b54141a10d7a43e5d21904, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:27.596412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654c9a0-2689-4c95-b280-a0628fac97ed None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:27.612136 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 132/273] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:27 2022] PUT /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d/users/b054fc3565b54141a10d7a43e5d21904/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:27.626504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9da67e-3149-49d6-851f-102027156a0a None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:27.755402 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9da67e-3149-49d6-851f-102027156a0a None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:27.795013 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400b98c-2b07-4fce-a6b9-bbaeadee5ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:27.795668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400b98c-2b07-4fce-a6b9-bbaeadee5ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:27.795948 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400b98c-2b07-4fce-a6b9-bbaeadee5ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:27.796223 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d400b98c-2b07-4fce-a6b9-bbaeadee5ad8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:27.797371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400b98c-2b07-4fce-a6b9-bbaeadee5ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:27.799422 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d400b98c-2b07-4fce-a6b9-bbaeadee5ad8 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:27.801412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 142/274] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:27.808119 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654c9a0-2689-4c95-b280-a0628fac97ed None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:27.900091 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400b98c-2b07-4fce-a6b9-bbaeadee5ad8 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:27.941310 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654c9a0-2689-4c95-b280-a0628fac97ed None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:27.941905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:27.994222 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2708a-1468-4b8c-b651-72d26ac10b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:27.995438 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2708a-1468-4b8c-b651-72d26ac10b38 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:27.995717 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2708a-1468-4b8c-b651-72d26ac10b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:27.995988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2708a-1468-4b8c-b651-72d26ac10b38 None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:27.996655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2708a-1468-4b8c-b651-72d26ac10b38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:27.998105 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2708a-1468-4b8c-b651-72d26ac10b38 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:28.021413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d400b98c-2b07-4fce-a6b9-bbaeadee5ad8 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:28.030787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 133/275] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:27 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:28.066935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c585d-a11a-4260-9d12-e7a14aea467d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:28.067612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c585d-a11a-4260-9d12-e7a14aea467d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:28.069129 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c585d-a11a-4260-9d12-e7a14aea467d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:28.069129 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c585d-a11a-4260-9d12-e7a14aea467d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:28.083180 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c585d-a11a-4260-9d12-e7a14aea467d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:28.086909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-509c585d-a11a-4260-9d12-e7a14aea467d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:28.086909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:28.086909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:28.086909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:28.086909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:28.086909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:28.086909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:28.086909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:28.086909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:28.086909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:28.086909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:28.086909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:28.086909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:28.086909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:28.086909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:28.086909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:28.087801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:28.087801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:28.087801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:28.087801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:28.087801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:28.087801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:28.087801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:28.087801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:28.087801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:28.087801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:28.087801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:28.087801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:28.087801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:28.087801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:28.087801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:28.087801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:28.089466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:28.089466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:28.089466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:28.089466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:28.089466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:28.089466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:28.089466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:28.089466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:28.089466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:28.089466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:28.089466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:28.089466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:28.089466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 143/276] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:28.094687 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2708a-1468-4b8c-b651-72d26ac10b38 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:28.116462 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:28.235031 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2708a-1468-4b8c-b651-72d26ac10b38 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:28.258763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5f73a-f7a0-40af-b3f6-8359db024353 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:28.259091 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5f73a-f7a0-40af-b3f6-8359db024353 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:28.259323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5f73a-f7a0-40af-b3f6-8359db024353 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:28.260791 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 144/277] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:44:28.266576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced20547-64af-4b34-813b-55e7a49eabaa None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:28.266886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced20547-64af-4b34-813b-55e7a49eabaa None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:28.267161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced20547-64af-4b34-813b-55e7a49eabaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:28.274021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958608e-683e-4704-af7e-434b27a1ee14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:28.274959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958608e-683e-4704-af7e-434b27a1ee14 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:28.275461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958608e-683e-4704-af7e-434b27a1ee14 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:28.275925 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958608e-683e-4704-af7e-434b27a1ee14 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:28.276822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958608e-683e-4704-af7e-434b27a1ee14 None admin] RBAC: Adding query filter params (name=member) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:28.277351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958608e-683e-4704-af7e-434b27a1ee14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:28.278750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958608e-683e-4704-af7e-434b27a1ee14 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:28.291622 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 134/278] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:28.292588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-ced20547-64af-4b34-813b-55e7a49eabaa None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:28.293872 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ced20547-64af-4b34-813b-55e7a49eabaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:28.299150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958608e-683e-4704-af7e-434b27a1ee14 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:28.381757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 145/279] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:28.398209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:28.442282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958608e-683e-4704-af7e-434b27a1ee14 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:28.484369 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c0169-12f2-4674-8d55-8792b9c3c8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:28.485011 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c0169-12f2-4674-8d55-8792b9c3c8ab None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:28.485296 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c0169-12f2-4674-8d55-8792b9c3c8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:28.485566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c0169-12f2-4674-8d55-8792b9c3c8ab None admin] PATH_INFO: `/v3/users/baff5d5755834c94ba5a52252028cebe` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:28.501535 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c0169-12f2-4674-8d55-8792b9c3c8ab None admin] RBAC: Authorizing `identity:get_user(user_id=baff5d5755834c94ba5a52252028cebe)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:28.503577 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c0169-12f2-4674-8d55-8792b9c3c8ab None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:28.505588 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 135/280] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:28 2022] GET /identity/v3/users/baff5d5755834c94ba5a52252028cebe => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:28.511396 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced20547-64af-4b34-813b-55e7a49eabaa None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:28.543030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:28.618932 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced20547-64af-4b34-813b-55e7a49eabaa None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:28.656781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a81889-a2a0-43ae-924e-d6f5eeee7db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:28.657823 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a81889-a2a0-43ae-924e-d6f5eeee7db1 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:28.658376 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a81889-a2a0-43ae-924e-d6f5eeee7db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:28.658908 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a81889-a2a0-43ae-924e-d6f5eeee7db1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:28.659837 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a81889-a2a0-43ae-924e-d6f5eeee7db1 None admin] RBAC: Adding query filter params (role.id=7d3cf58c1ad2407c9f8c60ba99b2006d, user.id=baff5d5755834c94ba5a52252028cebe, scope.system=all) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:28.660387 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a81889-a2a0-43ae-924e-d6f5eeee7db1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:28.662001 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a81889-a2a0-43ae-924e-d6f5eeee7db1 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:28.667725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 146/281] 10.222.0.26 () {56 vars in 1308 bytes} [Thu Sep 22 14:44:28 2022] GET /identity/v3/role_assignments?user.id=baff5d5755834c94ba5a52252028cebe&scope.system=all&role.id=7d3cf58c1ad2407c9f8c60ba99b2006d => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:28.702557 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c0169-12f2-4674-8d55-8792b9c3c8ab None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:28.854100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c0169-12f2-4674-8d55-8792b9c3c8ab None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:28.901362 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:28.909209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bfc8d-ea53-47dc-b156-4b1953c2acb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:28.909856 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bfc8d-ea53-47dc-b156-4b1953c2acb3 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:28.910132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bfc8d-ea53-47dc-b156-4b1953c2acb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:28.910402 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8bfc8d-ea53-47dc-b156-4b1953c2acb3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:28.927885 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8bfc8d-ea53-47dc-b156-4b1953c2acb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:28.932849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-4c8bfc8d-ea53-47dc-b156-4b1953c2acb3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:28.932849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:28.932849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:28.932849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:28.932849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:28.932849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:28.932849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:28.932849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:28.932849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:28.932849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:28.932849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:28.932849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:28.932849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:28.932849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:28.932849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:28.932849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:28.933748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:28.933748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:28.933748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:28.933748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:28.933748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:28.933748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:28.933748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:28.933748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:28.933748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:28.933748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:28.933748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:28.933748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:28.933748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:28.933748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:28.933748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:28.933748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:28.934988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:28.934988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:28.934988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:28.934988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:28.934988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:28.934988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:28.934988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:28.934988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:28.934988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:28.934988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:28.934988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:28.934988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:28.934988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 136/282] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:28.940918 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a81889-a2a0-43ae-924e-d6f5eeee7db1 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:29.051958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:29.060956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a81889-a2a0-43ae-924e-d6f5eeee7db1 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:29.095398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea485441-f926-4f1e-a5b5-c41d2508a35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:29.096892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea485441-f926-4f1e-a5b5-c41d2508a35d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:29.097215 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea485441-f926-4f1e-a5b5-c41d2508a35d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:29.097432 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea485441-f926-4f1e-a5b5-c41d2508a35d None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:29.098096 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea485441-f926-4f1e-a5b5-c41d2508a35d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:29.098393 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea485441-f926-4f1e-a5b5-c41d2508a35d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:29.099941 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea485441-f926-4f1e-a5b5-c41d2508a35d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:29.108927 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 147/283] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:29.114480 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bfc8d-ea53-47dc-b156-4b1953c2acb3 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:29.224144 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8bfc8d-ea53-47dc-b156-4b1953c2acb3 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:29.260028 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ff61b-ee01-45fa-8c2e-36aa58ac692c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:29.260718 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ff61b-ee01-45fa-8c2e-36aa58ac692c None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:29.260996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ff61b-ee01-45fa-8c2e-36aa58ac692c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:29.261265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ff61b-ee01-45fa-8c2e-36aa58ac692c None admin] PATH_INFO: `/v3/users/d8e797526d17487da511844ebbdb19db` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:29.262448 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ff61b-ee01-45fa-8c2e-36aa58ac692c None admin] RBAC: Authorizing `identity:get_user(user_id=d8e797526d17487da511844ebbdb19db)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:29.263971 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ff61b-ee01-45fa-8c2e-36aa58ac692c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:29.265538 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 137/284] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:29 2022] GET /identity/v3/users/d8e797526d17487da511844ebbdb19db => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:29.271963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea485441-f926-4f1e-a5b5-c41d2508a35d None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:29.395328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea485441-f926-4f1e-a5b5-c41d2508a35d None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:29.435719 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905f7df-d697-488c-84bb-6a5b40efc31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:29.436365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905f7df-d697-488c-84bb-6a5b40efc31f None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:29.436642 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905f7df-d697-488c-84bb-6a5b40efc31f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:29.436908 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905f7df-d697-488c-84bb-6a5b40efc31f None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:29.438065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905f7df-d697-488c-84bb-6a5b40efc31f None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:29.440139 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905f7df-d697-488c-84bb-6a5b40efc31f None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:29.441785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 148/285] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:29 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:29.447732 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ff61b-ee01-45fa-8c2e-36aa58ac692c None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:29.530150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:29.579571 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ff61b-ee01-45fa-8c2e-36aa58ac692c None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:29.615791 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69887a-619b-4190-af9f-9a13ee783a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:29.616450 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69887a-619b-4190-af9f-9a13ee783a65 None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:29.616724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69887a-619b-4190-af9f-9a13ee783a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:29.616992 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69887a-619b-4190-af9f-9a13ee783a65 None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d/users/d8e797526d17487da511844ebbdb19db/roles/77e7991dcd664ba086355ef20ff81541` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:29.618986 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69887a-619b-4190-af9f-9a13ee783a65 None admin] RBAC: Authorizing `identity:create_grant(project_id=45e79ea1241346c58bf04c1494b3dc7d, user_id=d8e797526d17487da511844ebbdb19db, role_id=77e7991dcd664ba086355ef20ff81541)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:29.620416 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69887a-619b-4190-af9f-9a13ee783a65 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:29.640598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 138/286] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:44:29 2022] PUT /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d/users/d8e797526d17487da511844ebbdb19db/roles/77e7991dcd664ba086355ef20ff81541 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:30.326927 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309fe59-3e3d-4236-b3f5-e12ca6ce8620 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:30.326927 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309fe59-3e3d-4236-b3f5-e12ca6ce8620 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:30.327271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309fe59-3e3d-4236-b3f5-e12ca6ce8620 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:30.328544 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 149/287] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:30.334718 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf4b44-9bed-4a9e-8beb-bbc099ebc3c2 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:30.335190 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf4b44-9bed-4a9e-8beb-bbc099ebc3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:30.335268 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf4b44-9bed-4a9e-8beb-bbc099ebc3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:30.354772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-83bf4b44-9bed-4a9e-8beb-bbc099ebc3c2 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:30.355965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-83bf4b44-9bed-4a9e-8beb-bbc099ebc3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:30.427449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 139/288] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:30.706955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309fe59-3e3d-4236-b3f5-e12ca6ce8620 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:30.738449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:30.824919 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309fe59-3e3d-4236-b3f5-e12ca6ce8620 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:30.867966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6b3f2-e612-4ccf-8130-f532bf19dbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:30.868605 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6b3f2-e612-4ccf-8130-f532bf19dbc2 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:30.868880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6b3f2-e612-4ccf-8130-f532bf19dbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:30.869151 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6b3f2-e612-4ccf-8130-f532bf19dbc2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:30.885446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6b3f2-e612-4ccf-8130-f532bf19dbc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:30.891188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-afd6b3f2-e612-4ccf-8130-f532bf19dbc2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:30.891188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:30.891188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:30.891188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:30.891188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:30.891188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:30.891188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:30.891188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:30.891188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:30.891188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:30.891188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:30.891188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:30.891188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:30.891188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:30.891188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:30.891188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:30.892097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:30.892097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:30.892097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:30.892097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:30.892097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:30.892097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:30.892097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:30.892097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:30.892097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:30.892097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:30.892097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:30.892097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:30.892097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:30.892097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:30.892097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:30.892097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:30.893501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:30.893501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:30.893501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:30.893501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:30.893501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:30.893501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:30.893501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:30.893501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:30.893501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:30.893501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:30.893501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:30.893501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:30.893501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 150/289] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:30.898902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf4b44-9bed-4a9e-8beb-bbc099ebc3c2 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:30.950469 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:31.011821 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf4b44-9bed-4a9e-8beb-bbc099ebc3c2 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:31.047490 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042692c3-6d02-43d1-94b9-a5b397de984e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:31.048109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042692c3-6d02-43d1-94b9-a5b397de984e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.048381 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042692c3-6d02-43d1-94b9-a5b397de984e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.049251 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042692c3-6d02-43d1-94b9-a5b397de984e None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.049901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042692c3-6d02-43d1-94b9-a5b397de984e None admin] RBAC: Adding query filter params (name=member) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:31.050199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042692c3-6d02-43d1-94b9-a5b397de984e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:31.051600 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042692c3-6d02-43d1-94b9-a5b397de984e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:31.065792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 140/290] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:31.067244 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6b3f2-e612-4ccf-8130-f532bf19dbc2 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:31.109317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098addb-82e9-414a-96ed-8d9703c2c4af None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.109648 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098addb-82e9-414a-96ed-8d9703c2c4af None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.109856 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7098addb-82e9-414a-96ed-8d9703c2c4af None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.111515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 141/291] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:31.120642 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf737a8-fed5-46f0-afbc-e5097fffcd76 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.120917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf737a8-fed5-46f0-afbc-e5097fffcd76 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.121182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf737a8-fed5-46f0-afbc-e5097fffcd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.149094 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-9cf737a8-fed5-46f0-afbc-e5097fffcd76 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:31.151021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-9cf737a8-fed5-46f0-afbc-e5097fffcd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:31.223161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:31.267091 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6b3f2-e612-4ccf-8130-f532bf19dbc2 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:31.294189 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 142/292] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:31.321057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc3be9-a409-48d5-a6c1-486c3cf7df59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:31.321701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc3be9-a409-48d5-a6c1-486c3cf7df59 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.321974 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc3be9-a409-48d5-a6c1-486c3cf7df59 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.322241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc3be9-a409-48d5-a6c1-486c3cf7df59 None admin] PATH_INFO: `/v3/users/b054fc3565b54141a10d7a43e5d21904` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.329426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc3be9-a409-48d5-a6c1-486c3cf7df59 None admin] RBAC: Authorizing `identity:get_user(user_id=b054fc3565b54141a10d7a43e5d21904)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:31.331605 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc3be9-a409-48d5-a6c1-486c3cf7df59 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:31.333684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 151/293] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:31 2022] GET /identity/v3/users/b054fc3565b54141a10d7a43e5d21904 => generated 297 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:31.339524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf737a8-fed5-46f0-afbc-e5097fffcd76 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:31.385212 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfbbe9-abd0-4145-9c33-779740aa3c86 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.385833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfbbe9-abd0-4145-9c33-779740aa3c86 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.386338 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfbbe9-abd0-4145-9c33-779740aa3c86 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.388159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 152/294] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:44:31.392120 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:31.394181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449259a-d741-493f-a7dc-6e8d0d6bbc44 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.394688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449259a-d741-493f-a7dc-6e8d0d6bbc44 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.395160 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449259a-d741-493f-a7dc-6e8d0d6bbc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.414212 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-3449259a-d741-493f-a7dc-6e8d0d6bbc44 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:31.415802 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3449259a-d741-493f-a7dc-6e8d0d6bbc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:31.448920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf737a8-fed5-46f0-afbc-e5097fffcd76 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:31.487503 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 153/295] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:31.501117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c54b86-0b31-454c-985e-d930b94e4bb6 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.502284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c54b86-0b31-454c-985e-d930b94e4bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.502763 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a90e8-4f71-40c5-8396-9f5c32c6b1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:31.502915 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c54b86-0b31-454c-985e-d930b94e4bb6 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.503659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a90e8-4f71-40c5-8396-9f5c32c6b1c2 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.504071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a90e8-4f71-40c5-8396-9f5c32c6b1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.504489 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 154/296] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:44:31.504619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a90e8-4f71-40c5-8396-9f5c32c6b1c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.512290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada03488-f42d-4a85-b91f-ab3378cc2a88 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.512917 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada03488-f42d-4a85-b91f-ab3378cc2a88 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.513395 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada03488-f42d-4a85-b91f-ab3378cc2a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.547998 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-ada03488-f42d-4a85-b91f-ab3378cc2a88 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:31.549571 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ada03488-f42d-4a85-b91f-ab3378cc2a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:31.594793 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a90e8-4f71-40c5-8396-9f5c32c6b1c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:31.596688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a90e8-4f71-40c5-8396-9f5c32c6b1c2 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:31.642139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:31.648119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 155/297] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:31.650114 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada03488-f42d-4a85-b91f-ab3378cc2a88 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:31.663417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:31.717283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 143/298] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:31.723407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a90e8-4f71-40c5-8396-9f5c32c6b1c2 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:31.765667 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada03488-f42d-4a85-b91f-ab3378cc2a88 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:31.824070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec4fff-5e7f-4b1e-b47b-05ada4972d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:31.824704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec4fff-5e7f-4b1e-b47b-05ada4972d82 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.824976 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec4fff-5e7f-4b1e-b47b-05ada4972d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.825241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec4fff-5e7f-4b1e-b47b-05ada4972d82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.827032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec4fff-5e7f-4b1e-b47b-05ada4972d82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:31.831260 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec4fff-5e7f-4b1e-b47b-05ada4972d82 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:31.841339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 156/299] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:31.848105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec4fff-5e7f-4b1e-b47b-05ada4972d82 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:31.914828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a90e8-4f71-40c5-8396-9f5c32c6b1c2 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:31.952640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:31.956976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a0714-bc85-4adb-98e8-6bae8863b832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:31.957967 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a0714-bc85-4adb-98e8-6bae8863b832 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:31.958448 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a0714-bc85-4adb-98e8-6bae8863b832 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:31.958943 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a0714-bc85-4adb-98e8-6bae8863b832 None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:31.961206 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a0714-bc85-4adb-98e8-6bae8863b832 None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:31.963596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a0714-bc85-4adb-98e8-6bae8863b832 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:31.965816 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 144/300] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:31 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:31.966879 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a0714-bc85-4adb-98e8-6bae8863b832 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:31.976221 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec4fff-5e7f-4b1e-b47b-05ada4972d82 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:31.979380 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:32.014654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a6513-26be-4b5b-9b04-97589526de8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:32.015294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a6513-26be-4b5b-9b04-97589526de8e None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:32.015631 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a6513-26be-4b5b-9b04-97589526de8e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:32.016109 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a6513-26be-4b5b-9b04-97589526de8e None admin] PATH_INFO: `/v3/groups` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:32.016535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a6513-26be-4b5b-9b04-97589526de8e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:32.018208 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a6513-26be-4b5b-9b04-97589526de8e None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:32.038903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 157/301] 10.222.0.26 () {60 vars in 1134 bytes} [Thu Sep 22 14:44:31 2022] POST /identity/v3/groups => generated 229 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:44:32.039804 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a6513-26be-4b5b-9b04-97589526de8e None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:32.086854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a0714-bc85-4adb-98e8-6bae8863b832 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:32.131429 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b5982-4646-4dbe-88cd-d485e7cb314e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:32.132388 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b5982-4646-4dbe-88cd-d485e7cb314e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:32.132388 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b5982-4646-4dbe-88cd-d485e7cb314e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:32.132695 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b5982-4646-4dbe-88cd-d485e7cb314e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:32.146956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b5982-4646-4dbe-88cd-d485e7cb314e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:32.152314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-563b5982-4646-4dbe-88cd-d485e7cb314e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:32.152314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:32.152314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:32.152314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:32.152314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:32.152314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:32.152314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:32.152314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:32.152314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:32.152314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:32.152314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:32.152314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:32.152314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:32.152314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:32.152314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:32.152314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:32.153344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:32.153344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:32.153344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:32.153344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:32.153344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:32.153344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:32.153344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:32.153344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:32.153344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:32.153344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:32.153344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:32.153344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:32.153344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:32.153344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:32.153344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:32.153344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:32.154152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:32.154152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:32.154152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:32.154152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:32.154152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:32.154152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:32.154152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:32.154152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:32.154152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:32.154152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:32.154152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:32.154152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:32.154152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 145/302] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:32.154839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b5982-4646-4dbe-88cd-d485e7cb314e None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:32.218243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a6513-26be-4b5b-9b04-97589526de8e None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:32.236948 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:32.262406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae640a55-26ac-4603-8708-2525aa9c3b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:32.263235 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae640a55-26ac-4603-8708-2525aa9c3b08 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:32.263518 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae640a55-26ac-4603-8708-2525aa9c3b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:32.263785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae640a55-26ac-4603-8708-2525aa9c3b08 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:32.272392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b5982-4646-4dbe-88cd-d485e7cb314e None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:32.272958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:32.278359 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae640a55-26ac-4603-8708-2525aa9c3b08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:32.284576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-ae640a55-26ac-4603-8708-2525aa9c3b08 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:32.284576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:32.284576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:32.284576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:32.284576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:32.284576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:32.284576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:32.284576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:32.284576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:32.284576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:32.284576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:32.284576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:32.284576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:32.284576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:32.284576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:32.284576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:32.285463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:32.285463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:32.285463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:32.285463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:32.285463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:32.285463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:32.285463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:32.285463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:32.285463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:32.285463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:32.285463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:32.285463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:32.285463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:32.285463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:32.285463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:32.285463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:32.286254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:32.286254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:32.286254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:32.286254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:32.286254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:32.286254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:32.286254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:32.286254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:32.286254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:32.286254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:32.286254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:32.286254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:32.288829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 158/303] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:32.290174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae640a55-26ac-4603-8708-2525aa9c3b08 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:32.322501 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15db6df-c067-4fd9-9bb4-f523df6623bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:32.323191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15db6df-c067-4fd9-9bb4-f523df6623bd None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:32.323470 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15db6df-c067-4fd9-9bb4-f523df6623bd None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:32.323740 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15db6df-c067-4fd9-9bb4-f523df6623bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:32.324479 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15db6df-c067-4fd9-9bb4-f523df6623bd None admin] RBAC: Adding query filter params (role.id=7d3cf58c1ad2407c9f8c60ba99b2006d, scope.project.id=45e79ea1241346c58bf04c1494b3dc7d, user.id=b054fc3565b54141a10d7a43e5d21904) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:32.324785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15db6df-c067-4fd9-9bb4-f523df6623bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:32.326247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15db6df-c067-4fd9-9bb4-f523df6623bd None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:32.333082 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 146/304] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:32 2022] GET /identity/v3/role_assignments?user.id=b054fc3565b54141a10d7a43e5d21904&scope.project.id=45e79ea1241346c58bf04c1494b3dc7d&role.id=7d3cf58c1ad2407c9f8c60ba99b2006d => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:32.333742 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15db6df-c067-4fd9-9bb4-f523df6623bd None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:32.473360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15db6df-c067-4fd9-9bb4-f523df6623bd None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:32.510334 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:32.519452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73266d-42dd-467f-86a8-57ed987ac28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:32.522188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73266d-42dd-467f-86a8-57ed987ac28f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:32.528342 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73266d-42dd-467f-86a8-57ed987ac28f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:32.528923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73266d-42dd-467f-86a8-57ed987ac28f None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:32.529882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73266d-42dd-467f-86a8-57ed987ac28f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:32.530412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73266d-42dd-467f-86a8-57ed987ac28f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:32.532237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73266d-42dd-467f-86a8-57ed987ac28f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:32.553371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 147/305] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:32.560246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73266d-42dd-467f-86a8-57ed987ac28f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:32.579106 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae640a55-26ac-4603-8708-2525aa9c3b08 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:32.621588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b3079-6432-457d-a983-52c7b1b9c2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:32.622311 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b3079-6432-457d-a983-52c7b1b9c2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:32.622592 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b3079-6432-457d-a983-52c7b1b9c2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:32.622989 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b3079-6432-457d-a983-52c7b1b9c2d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:32.623674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b3079-6432-457d-a983-52c7b1b9c2d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:32.623994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b3079-6432-457d-a983-52c7b1b9c2d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:32.625693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b3079-6432-457d-a983-52c7b1b9c2d7 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:32.636259 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 159/306] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:32.637521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363231a-4254-45e6-af79-375c967f910e None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:32.637814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363231a-4254-45e6-af79-375c967f910e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:32.638091 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363231a-4254-45e6-af79-375c967f910e None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:32.639903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 160/307] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:32.643724 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363231a-4254-45e6-af79-375c967f910e None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:32.698718 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73266d-42dd-467f-86a8-57ed987ac28f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:32.726628 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:32.735808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc58813-21a3-49ed-b711-73bcb6162642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:32.736377 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc58813-21a3-49ed-b711-73bcb6162642 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:32.736658 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc58813-21a3-49ed-b711-73bcb6162642 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:32.736929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc58813-21a3-49ed-b711-73bcb6162642 None admin] PATH_INFO: `/v3/users/24bfa515767042c382ce7147747a5eaa` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:32.752726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc58813-21a3-49ed-b711-73bcb6162642 None admin] RBAC: Authorizing `identity:get_user(user_id=24bfa515767042c382ce7147747a5eaa)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:32.754377 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc58813-21a3-49ed-b711-73bcb6162642 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:32.761289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 148/308] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:32 2022] GET /identity/v3/users/24bfa515767042c382ce7147747a5eaa => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:32.761289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60757d-d69b-40e9-a7ff-8ea8f0b7e378 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:32.761289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60757d-d69b-40e9-a7ff-8ea8f0b7e378 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:32.761289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60757d-d69b-40e9-a7ff-8ea8f0b7e378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:32.774849 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363231a-4254-45e6-af79-375c967f910e None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:32.776282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-4d60757d-d69b-40e9-a7ff-8ea8f0b7e378 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:32.777466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4d60757d-d69b-40e9-a7ff-8ea8f0b7e378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:32.817293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2a2f1-a6df-49d1-aa60-f574bae8006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:32.817936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2a2f1-a6df-49d1-aa60-f574bae8006b None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:32.818224 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2a2f1-a6df-49d1-aa60-f574bae8006b None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:32.818499 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2a2f1-a6df-49d1-aa60-f574bae8006b None admin] PATH_INFO: `/v3/users/baff5d5755834c94ba5a52252028cebe` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:32.819673 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c2a2f1-a6df-49d1-aa60-f574bae8006b None admin] RBAC: Authorizing `identity:get_user(user_id=baff5d5755834c94ba5a52252028cebe)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:32.821582 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c2a2f1-a6df-49d1-aa60-f574bae8006b None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:32.823846 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 161/309] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:32 2022] GET /identity/v3/users/baff5d5755834c94ba5a52252028cebe => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:32.824402 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2a2f1-a6df-49d1-aa60-f574bae8006b None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:32.828588 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:32.853330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 149/310] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:32.854164 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60757d-d69b-40e9-a7ff-8ea8f0b7e378 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:32.940564 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2a2f1-a6df-49d1-aa60-f574bae8006b None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:32.976069 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:32.984988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60757d-d69b-40e9-a7ff-8ea8f0b7e378 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:32.992374 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece3625-c338-4944-99f5-2e3fe87c008b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:32.993378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece3625-c338-4944-99f5-2e3fe87c008b None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:32.993860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece3625-c338-4944-99f5-2e3fe87c008b None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:32.994354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece3625-c338-4944-99f5-2e3fe87c008b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:32.995424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece3625-c338-4944-99f5-2e3fe87c008b None admin] RBAC: Adding query filter params (role.id=77e7991dcd664ba086355ef20ff81541, user.id=24bfa515767042c382ce7147747a5eaa, scope.system=all) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:32.995936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece3625-c338-4944-99f5-2e3fe87c008b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:32.997705 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece3625-c338-4944-99f5-2e3fe87c008b None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:33.004388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 162/311] 10.222.0.26 () {56 vars in 1308 bytes} [Thu Sep 22 14:44:32 2022] GET /identity/v3/role_assignments?user.id=24bfa515767042c382ce7147747a5eaa&scope.system=all&role.id=77e7991dcd664ba086355ef20ff81541 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:33.024471 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b469b-277b-4bf5-8d76-45346d202fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:33.025161 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b469b-277b-4bf5-8d76-45346d202fe6 None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:33.025498 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b469b-277b-4bf5-8d76-45346d202fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:33.025772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b469b-277b-4bf5-8d76-45346d202fe6 None admin] PATH_INFO: `/v3/system/users/baff5d5755834c94ba5a52252028cebe/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:33.027406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b469b-277b-4bf5-8d76-45346d202fe6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=baff5d5755834c94ba5a52252028cebe, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:33.028898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b469b-277b-4bf5-8d76-45346d202fe6 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:33.039044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 150/312] 10.222.0.26 () {58 vars in 1334 bytes} [Thu Sep 22 14:44:32 2022] PUT /identity/v3/system/users/baff5d5755834c94ba5a52252028cebe/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:33.140712 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece3625-c338-4944-99f5-2e3fe87c008b None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:33.254819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece3625-c338-4944-99f5-2e3fe87c008b None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:33.288568 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2feef1-368a-477c-8dbc-9a6c6e53793e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:33.289291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2feef1-368a-477c-8dbc-9a6c6e53793e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:33.289565 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2feef1-368a-477c-8dbc-9a6c6e53793e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:33.289830 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2feef1-368a-477c-8dbc-9a6c6e53793e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:33.300669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2feef1-368a-477c-8dbc-9a6c6e53793e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:33.306022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-9c2feef1-368a-477c-8dbc-9a6c6e53793e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:33.306022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:33.306022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:33.306022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:33.306022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:33.306022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:33.306022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:33.306022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:33.306022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:33.306022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:33.306022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:33.306022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:33.306022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:33.306022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:33.306022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:33.306022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:33.306966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:33.306966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:33.306966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:33.306966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:33.306966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:33.306966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:33.306966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:33.306966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:33.306966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:33.306966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:33.306966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:33.306966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:33.306966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:33.306966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:33.306966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:33.306966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:33.308185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:33.308185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:33.308185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:33.308185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:33.308185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:33.308185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:33.308185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:33.308185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:33.308185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:33.308185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:33.308185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:33.308185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:33.308185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 163/313] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:33.315563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b469b-277b-4bf5-8d76-45346d202fe6 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:33.367381 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:33.428242 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b469b-277b-4bf5-8d76-45346d202fe6 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:33.465654 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8e0ef-74d0-4b12-8bf1-47543c66b922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:33.466265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8e0ef-74d0-4b12-8bf1-47543c66b922 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:33.466542 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8e0ef-74d0-4b12-8bf1-47543c66b922 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:33.466839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8e0ef-74d0-4b12-8bf1-47543c66b922 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:33.467512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a8e0ef-74d0-4b12-8bf1-47543c66b922 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:33.467811 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a8e0ef-74d0-4b12-8bf1-47543c66b922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:33.469192 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a8e0ef-74d0-4b12-8bf1-47543c66b922 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:33.479430 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 151/314] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:33.485697 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2feef1-368a-477c-8dbc-9a6c6e53793e None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:33.543323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:33.599118 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2feef1-368a-477c-8dbc-9a6c6e53793e None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:33.633872 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236e3ef-1231-4096-afc7-d7cc5d11daf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:33.634514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236e3ef-1231-4096-afc7-d7cc5d11daf1 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:33.634816 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236e3ef-1231-4096-afc7-d7cc5d11daf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:33.635091 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236e3ef-1231-4096-afc7-d7cc5d11daf1 None admin] PATH_INFO: `/v3/users/d8e797526d17487da511844ebbdb19db` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:33.636243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236e3ef-1231-4096-afc7-d7cc5d11daf1 None admin] RBAC: Authorizing `identity:get_user(user_id=d8e797526d17487da511844ebbdb19db)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:33.637684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236e3ef-1231-4096-afc7-d7cc5d11daf1 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:33.644114 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 164/315] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:33 2022] GET /identity/v3/users/d8e797526d17487da511844ebbdb19db => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:33.645478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8e0ef-74d0-4b12-8bf1-47543c66b922 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:33.754384 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8e0ef-74d0-4b12-8bf1-47543c66b922 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:33.763986 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:33.791452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7e913-8ce9-4403-952b-d86f48dc8d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:33.792468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7e913-8ce9-4403-952b-d86f48dc8d40 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:33.792947 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7e913-8ce9-4403-952b-d86f48dc8d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:33.793473 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7e913-8ce9-4403-952b-d86f48dc8d40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:33.871856 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7e913-8ce9-4403-952b-d86f48dc8d40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:33.873667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7e913-8ce9-4403-952b-d86f48dc8d40 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:33.985506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 152/316] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:33.991664 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236e3ef-1231-4096-afc7-d7cc5d11daf1 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:34.101256 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236e3ef-1231-4096-afc7-d7cc5d11daf1 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:34.121618 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:34.137824 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8f2aa-045a-4153-a0f1-73ce0d0c2b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:34.138921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8f2aa-045a-4153-a0f1-73ce0d0c2b74 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:34.139530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8f2aa-045a-4153-a0f1-73ce0d0c2b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:34.140047 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8f2aa-045a-4153-a0f1-73ce0d0c2b74 None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:34.142018 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8f2aa-045a-4153-a0f1-73ce0d0c2b74 None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:34.144755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8f2aa-045a-4153-a0f1-73ce0d0c2b74 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:34.146743 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 165/317] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:33 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:34.155892 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7e913-8ce9-4403-952b-d86f48dc8d40 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:34.168953 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:34.270620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7e913-8ce9-4403-952b-d86f48dc8d40 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:34.306283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd7eef-48ab-49d8-95f8-d33defb4cae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:34.306927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd7eef-48ab-49d8-95f8-d33defb4cae5 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:34.307202 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd7eef-48ab-49d8-95f8-d33defb4cae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:34.307466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd7eef-48ab-49d8-95f8-d33defb4cae5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:34.308191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd7eef-48ab-49d8-95f8-d33defb4cae5 None admin] RBAC: Adding query filter params (role.id=77e7991dcd664ba086355ef20ff81541, scope.project.id=45e79ea1241346c58bf04c1494b3dc7d, user.id=d8e797526d17487da511844ebbdb19db) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:34.308492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd7eef-48ab-49d8-95f8-d33defb4cae5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:34.309922 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd7eef-48ab-49d8-95f8-d33defb4cae5 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:34.316518 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 153/318] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:44:34 2022] GET /identity/v3/role_assignments?user.id=d8e797526d17487da511844ebbdb19db&scope.project.id=45e79ea1241346c58bf04c1494b3dc7d&role.id=77e7991dcd664ba086355ef20ff81541 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:34.451574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffbaed-d877-4cb7-b22e-ba978a5bc88e None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:34.451765 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffbaed-d877-4cb7-b22e-ba978a5bc88e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:34.451981 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffbaed-d877-4cb7-b22e-ba978a5bc88e None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:34.453666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 166/319] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:34.459912 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea074a3-6b4b-43ef-8469-cddc5bc052fa None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:34.460191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea074a3-6b4b-43ef-8469-cddc5bc052fa None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:34.460456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea074a3-6b4b-43ef-8469-cddc5bc052fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:34.480220 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-2ea074a3-6b4b-43ef-8469-cddc5bc052fa None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:34.481431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2ea074a3-6b4b-43ef-8469-cddc5bc052fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:34.557991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 154/320] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:34.825695 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffbaed-d877-4cb7-b22e-ba978a5bc88e None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:34.936179 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffbaed-d877-4cb7-b22e-ba978a5bc88e None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:34.970188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e80efc-8dd2-4043-b7dc-d69b1f6e6502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:34.970992 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e80efc-8dd2-4043-b7dc-d69b1f6e6502 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:34.971542 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e80efc-8dd2-4043-b7dc-d69b1f6e6502 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:34.971807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e80efc-8dd2-4043-b7dc-d69b1f6e6502 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:34.972902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e80efc-8dd2-4043-b7dc-d69b1f6e6502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:34.974320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e80efc-8dd2-4043-b7dc-d69b1f6e6502 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:34.976117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 167/321] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:34.981457 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea074a3-6b4b-43ef-8469-cddc5bc052fa None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:34.983681 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:35.088907 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea074a3-6b4b-43ef-8469-cddc5bc052fa None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:35.124404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c83b17-8ca8-417e-9fcd-727918f8ba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:35.125073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c83b17-8ca8-417e-9fcd-727918f8ba7c None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:35.125357 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c83b17-8ca8-417e-9fcd-727918f8ba7c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:35.125687 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c83b17-8ca8-417e-9fcd-727918f8ba7c None admin] PATH_INFO: `/v3/groups` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:35.126366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c83b17-8ca8-417e-9fcd-727918f8ba7c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:35.128532 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c83b17-8ca8-417e-9fcd-727918f8ba7c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:35.147955 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 155/322] 10.222.0.26 () {60 vars in 1134 bytes} [Thu Sep 22 14:44:34 2022] POST /identity/v3/groups => generated 226 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:35.274941 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be97a5-fdb7-41f2-971c-6bc9a1a57e8c None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:35.275465 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be97a5-fdb7-41f2-971c-6bc9a1a57e8c None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:35.275579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be97a5-fdb7-41f2-971c-6bc9a1a57e8c None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:35.277122 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 168/323] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:35.282705 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19e658-79f9-4e42-975e-5fbe98b56bab None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:35.283556 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19e658-79f9-4e42-975e-5fbe98b56bab None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:35.283826 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19e658-79f9-4e42-975e-5fbe98b56bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:35.304137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-5d19e658-79f9-4e42-975e-5fbe98b56bab None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:35.305363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-5d19e658-79f9-4e42-975e-5fbe98b56bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:35.378494 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:35.390968 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 156/324] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:35.448823 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf05597-cfdd-4757-b243-4b8cc3825afc None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:35.449372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf05597-cfdd-4757-b243-4b8cc3825afc None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:35.449794 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf05597-cfdd-4757-b243-4b8cc3825afc None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:35.451326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 169/325] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:35.457008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84539f20-d6bc-45b3-afc1-2a9035f61a26 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:35.457309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84539f20-d6bc-45b3-afc1-2a9035f61a26 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:35.457545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84539f20-d6bc-45b3-afc1-2a9035f61a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:35.477275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-84539f20-d6bc-45b3-afc1-2a9035f61a26 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:35.478445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-84539f20-d6bc-45b3-afc1-2a9035f61a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:35.550033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 157/326] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:35.667362 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf05597-cfdd-4757-b243-4b8cc3825afc None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:35.747213 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:35.799862 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf05597-cfdd-4757-b243-4b8cc3825afc None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:35.823330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84539f20-d6bc-45b3-afc1-2a9035f61a26 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:35.845122 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec8d58-d12e-432e-98fb-bcb773285900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:35.846617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec8d58-d12e-432e-98fb-bcb773285900 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:35.847020 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec8d58-d12e-432e-98fb-bcb773285900 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:35.847309 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec8d58-d12e-432e-98fb-bcb773285900 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:35.861795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec8d58-d12e-432e-98fb-bcb773285900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:35.868451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-82ec8d58-d12e-432e-98fb-bcb773285900 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:35.868451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:35.868451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:35.868451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:35.868451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:35.868451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:35.868451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:35.868451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:35.868451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:35.868451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:35.868451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:35.868451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:35.868451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:35.868451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:35.868451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:35.868451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:35.869398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:35.869398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:35.869398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:35.869398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:35.869398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:35.869398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:35.869398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:35.869398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:35.869398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:35.869398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:35.869398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:35.869398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:35.869398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:35.869398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:35.869398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:35.869398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:35.870188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:35.870188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:35.870188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:35.870188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:35.870188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:35.870188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:35.870188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:35.870188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:35.870188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:35.870188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:35.870188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:35.870188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:35.870188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 170/327] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:35.875706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec8d58-d12e-432e-98fb-bcb773285900 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:35.981368 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84539f20-d6bc-45b3-afc1-2a9035f61a26 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:36.010684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec8d58-d12e-432e-98fb-bcb773285900 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:36.013934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:36.032028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:36.037973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e717621-7e7d-4f98-922e-0f5578ab47d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:36.039121 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e717621-7e7d-4f98-922e-0f5578ab47d7 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:36.039629 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e717621-7e7d-4f98-922e-0f5578ab47d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:36.041040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e717621-7e7d-4f98-922e-0f5578ab47d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:36.057157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e717621-7e7d-4f98-922e-0f5578ab47d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:36.059482 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b038876-2ddc-455e-852d-d7e9e14a698d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:36.060105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b038876-2ddc-455e-852d-d7e9e14a698d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:36.060383 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b038876-2ddc-455e-852d-d7e9e14a698d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:36.060651 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b038876-2ddc-455e-852d-d7e9e14a698d None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:36.063733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-0e717621-7e7d-4f98-922e-0f5578ab47d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:36.063733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:36.063733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:36.063733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:36.063733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:36.063733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:36.063733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:36.063733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:36.063733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:36.063733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:36.063733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:36.063733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:36.063733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:36.063733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:36.063733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:36.063733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:36.065147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:36.065147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:36.065147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:36.065147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:36.065147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:36.065147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:36.065147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:36.065147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:36.065147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:36.065147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:36.065147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:36.065147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:36.065147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:36.065147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:36.065147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:36.065147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:36.065957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:36.065957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:36.065957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:36.065957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:36.065957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:36.065957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:36.065957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:36.065957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:36.065957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:36.065957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:36.065957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:36.065957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:36.066558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b038876-2ddc-455e-852d-d7e9e14a698d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:36.066558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b038876-2ddc-455e-852d-d7e9e14a698d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:36.066558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b038876-2ddc-455e-852d-d7e9e14a698d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:36.069096 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 158/328] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:36.076855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e717621-7e7d-4f98-922e-0f5578ab47d7 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:36.082676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 171/329] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:36.088914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b038876-2ddc-455e-852d-d7e9e14a698d None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:36.212187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e717621-7e7d-4f98-922e-0f5578ab47d7 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:36.222280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b038876-2ddc-455e-852d-d7e9e14a698d None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:36.268367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216ca61-60f0-482e-acb0-f91c28215e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:36.270044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216ca61-60f0-482e-acb0-f91c28215e33 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:36.271268 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216ca61-60f0-482e-acb0-f91c28215e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:36.271752 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2216ca61-60f0-482e-acb0-f91c28215e33 None admin] PATH_INFO: `/v3/users/24bfa515767042c382ce7147747a5eaa` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:36.274484 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216ca61-60f0-482e-acb0-f91c28215e33 None admin] RBAC: Authorizing `identity:get_user(user_id=24bfa515767042c382ce7147747a5eaa)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:36.276602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2216ca61-60f0-482e-acb0-f91c28215e33 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:36.280507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6fde7-b7b0-40f0-a03d-4e33ab00d48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:36.281390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6fde7-b7b0-40f0-a03d-4e33ab00d48b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:36.281883 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6fde7-b7b0-40f0-a03d-4e33ab00d48b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:36.282352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6fde7-b7b0-40f0-a03d-4e33ab00d48b None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:36.283799 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6fde7-b7b0-40f0-a03d-4e33ab00d48b None admin] RBAC: Adding query filter params (name=member) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:36.284264 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 172/330] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:36 2022] GET /identity/v3/users/24bfa515767042c382ce7147747a5eaa => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:36.285259 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216ca61-60f0-482e-acb0-f91c28215e33 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:36.286149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6fde7-b7b0-40f0-a03d-4e33ab00d48b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:36.288111 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6fde7-b7b0-40f0-a03d-4e33ab00d48b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:36.309566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 159/331] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:36.315433 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6fde7-b7b0-40f0-a03d-4e33ab00d48b None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:36.382294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:36.387514 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:36.454222 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6fde7-b7b0-40f0-a03d-4e33ab00d48b None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:36.492791 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2216ca61-60f0-482e-acb0-f91c28215e33 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:36.498758 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc898869-a4ad-4690-be35-a0c037d566ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:36.498758 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc898869-a4ad-4690-be35-a0c037d566ec None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:36.498758 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc898869-a4ad-4690-be35-a0c037d566ec None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:36.498758 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc898869-a4ad-4690-be35-a0c037d566ec None admin] PATH_INFO: `/v3/users/baff5d5755834c94ba5a52252028cebe` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:36.499060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc898869-a4ad-4690-be35-a0c037d566ec None admin] RBAC: Authorizing `identity:get_user(user_id=baff5d5755834c94ba5a52252028cebe)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:36.499735 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc898869-a4ad-4690-be35-a0c037d566ec None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:36.503062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 160/332] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:36 2022] GET /identity/v3/users/baff5d5755834c94ba5a52252028cebe => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:36.512006 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc898869-a4ad-4690-be35-a0c037d566ec None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:36.560963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fa37e-7dc9-44d7-b5b5-82bbe4387fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:36.561999 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fa37e-7dc9-44d7-b5b5-82bbe4387fde None admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:36.562474 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fa37e-7dc9-44d7-b5b5-82bbe4387fde None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:36.562971 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fa37e-7dc9-44d7-b5b5-82bbe4387fde None admin] PATH_INFO: `/v3/system/users/24bfa515767042c382ce7147747a5eaa/roles/77e7991dcd664ba086355ef20ff81541` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:36.565398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545fa37e-7dc9-44d7-b5b5-82bbe4387fde None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=24bfa515767042c382ce7147747a5eaa, role_id=77e7991dcd664ba086355ef20ff81541)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:36.567151 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545fa37e-7dc9-44d7-b5b5-82bbe4387fde None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:36.587184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 173/333] 10.222.0.26 () {58 vars in 1334 bytes} [Thu Sep 22 14:44:36 2022] PUT /identity/v3/system/users/24bfa515767042c382ce7147747a5eaa/roles/77e7991dcd664ba086355ef20ff81541 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:36.633547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc898869-a4ad-4690-be35-a0c037d566ec None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:36.647891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:36.670630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100b1e20-d825-448e-a10f-5c9805dd8f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:36.671305 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b1e20-d825-448e-a10f-5c9805dd8f33 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:36.671584 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b1e20-d825-448e-a10f-5c9805dd8f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:36.671857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100b1e20-d825-448e-a10f-5c9805dd8f33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:36.672591 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b1e20-d825-448e-a10f-5c9805dd8f33 None admin] RBAC: Adding query filter params (role.id=7d3cf58c1ad2407c9f8c60ba99b2006d, user.id=baff5d5755834c94ba5a52252028cebe, scope.system=all) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:36.672900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b1e20-d825-448e-a10f-5c9805dd8f33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:36.674911 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100b1e20-d825-448e-a10f-5c9805dd8f33 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:36.688108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 161/334] 10.222.0.26 () {56 vars in 1308 bytes} [Thu Sep 22 14:44:36 2022] GET /identity/v3/role_assignments?user.id=baff5d5755834c94ba5a52252028cebe&scope.system=all&role.id=7d3cf58c1ad2407c9f8c60ba99b2006d => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:37.652094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf581a0-bda3-45d5-9fd9-891471d43058 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:37.653367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf581a0-bda3-45d5-9fd9-891471d43058 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:37.653865 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf581a0-bda3-45d5-9fd9-891471d43058 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:37.655595 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 174/335] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:37.663561 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db92ca-83ac-48f4-a33f-f74a31197f60 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:37.664558 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db92ca-83ac-48f4-a33f-f74a31197f60 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:37.665039 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db92ca-83ac-48f4-a33f-f74a31197f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:37.687078 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-f0db92ca-83ac-48f4-a33f-f74a31197f60 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:37.688633 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f0db92ca-83ac-48f4-a33f-f74a31197f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:37.767901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 162/336] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:38.228856 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf581a0-bda3-45d5-9fd9-891471d43058 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:38.367062 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf581a0-bda3-45d5-9fd9-891471d43058 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:38.379539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:38.408413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12efcd-75d8-47b7-81d1-25b6856afc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:38.409062 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12efcd-75d8-47b7-81d1-25b6856afc8e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:38.409341 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12efcd-75d8-47b7-81d1-25b6856afc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:38.409610 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12efcd-75d8-47b7-81d1-25b6856afc8e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:38.422589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12efcd-75d8-47b7-81d1-25b6856afc8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:38.428442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-6a12efcd-75d8-47b7-81d1-25b6856afc8e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:38.428442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:38.428442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:38.428442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:38.428442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:38.428442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:38.428442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:38.428442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:38.428442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:38.428442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:38.428442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:38.428442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:38.428442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:38.428442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:38.428442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:38.428442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:38.429577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:38.429577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:38.429577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:38.429577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:38.429577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:38.429577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:38.429577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:38.429577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:38.429577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:38.429577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:38.429577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:38.429577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:38.429577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:38.429577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:38.429577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:38.429577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:38.430409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:38.430409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:38.430409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:38.430409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:38.430409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:38.430409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:38.430409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:38.430409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:38.430409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:38.430409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:38.430409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:38.430409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:38.433022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 175/337] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:38.436156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db92ca-83ac-48f4-a33f-f74a31197f60 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:38.537097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c9446-5dc9-4029-a27f-43569117f568 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:38.537276 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c9446-5dc9-4029-a27f-43569117f568 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:38.537650 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c9446-5dc9-4029-a27f-43569117f568 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:38.539028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 176/338] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:44:38.546552 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714df50-df1f-4b1c-ba9f-a36a80ca4494 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:38.546860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714df50-df1f-4b1c-ba9f-a36a80ca4494 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:38.547136 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714df50-df1f-4b1c-ba9f-a36a80ca4494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:38.567107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:38.574293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db92ca-83ac-48f4-a33f-f74a31197f60 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:38.580337 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-f714df50-df1f-4b1c-ba9f-a36a80ca4494 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:38.581651 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-f714df50-df1f-4b1c-ba9f-a36a80ca4494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:38.635833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9cff7-d629-42a1-b5e7-cde8c41aa6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:38.636450 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9cff7-d629-42a1-b5e7-cde8c41aa6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:38.636771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9cff7-d629-42a1-b5e7-cde8c41aa6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:38.637046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9cff7-d629-42a1-b5e7-cde8c41aa6d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:38.637717 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9cff7-d629-42a1-b5e7-cde8c41aa6d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:38.638018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9cff7-d629-42a1-b5e7-cde8c41aa6d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:38.639579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9cff7-d629-42a1-b5e7-cde8c41aa6d0 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:38.654594 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 163/339] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:38.661569 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9cff7-d629-42a1-b5e7-cde8c41aa6d0 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:38.713433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 177/340] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:38.805945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9cff7-d629-42a1-b5e7-cde8c41aa6d0 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:38.846682 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64698f5f-dac4-4e74-b151-3bb20a983b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:38.847313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64698f5f-dac4-4e74-b151-3bb20a983b78 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:38.847591 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64698f5f-dac4-4e74-b151-3bb20a983b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:38.847862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64698f5f-dac4-4e74-b151-3bb20a983b78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:38.886561 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:38.932467 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64698f5f-dac4-4e74-b151-3bb20a983b78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:38.933934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64698f5f-dac4-4e74-b151-3bb20a983b78 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:39.049516 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 164/341] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 14:44:39.055859 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714df50-df1f-4b1c-ba9f-a36a80ca4494 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:39.195868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:39.199173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714df50-df1f-4b1c-ba9f-a36a80ca4494 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:39.264421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2855e6-37de-459a-be5f-05a1336b8e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:39.265077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2855e6-37de-459a-be5f-05a1336b8e8a None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:39.265355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2855e6-37de-459a-be5f-05a1336b8e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:39.265626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2855e6-37de-459a-be5f-05a1336b8e8a None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:39.271470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2855e6-37de-459a-be5f-05a1336b8e8a None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:39.272936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2855e6-37de-459a-be5f-05a1336b8e8a None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:39.275076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 178/342] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:39 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:39.280344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64698f5f-dac4-4e74-b151-3bb20a983b78 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:39.364952 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2855e6-37de-459a-be5f-05a1336b8e8a None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:39.469651 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:39.588972 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64698f5f-dac4-4e74-b151-3bb20a983b78 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:39.621337 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2855e6-37de-459a-be5f-05a1336b8e8a None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:39.691463 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ad3a2-7f9a-42a2-b10e-a853a6ded3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:39.692495 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ad3a2-7f9a-42a2-b10e-a853a6ded3c4 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:39.692978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ad3a2-7f9a-42a2-b10e-a853a6ded3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:39.693495 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ad3a2-7f9a-42a2-b10e-a853a6ded3c4 None admin] PATH_INFO: `/v3/groups/f7bfc768836142d0b409afa8b8c12313` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:39.695451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2ef32-ccb1-49fc-9fa6-b516c749d307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:39.696098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2ef32-ccb1-49fc-9fa6-b516c749d307 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:39.696430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2ef32-ccb1-49fc-9fa6-b516c749d307 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:39.696700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2ef32-ccb1-49fc-9fa6-b516c749d307 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:39.707381 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ad3a2-7f9a-42a2-b10e-a853a6ded3c4 None admin] RBAC: Authorizing `identity:get_group(group_id=f7bfc768836142d0b409afa8b8c12313)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:39.709263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ad3a2-7f9a-42a2-b10e-a853a6ded3c4 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:39.711519 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 165/343] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:39 2022] GET /identity/v3/groups/f7bfc768836142d0b409afa8b8c12313 => generated 226 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:39.716208 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd2ef32-ccb1-49fc-9fa6-b516c749d307 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:39.721782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-dcd2ef32-ccb1-49fc-9fa6-b516c749d307 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:39.721782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:39.721782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:39.721782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:39.721782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:39.721782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:39.721782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:39.721782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:39.721782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:39.721782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:39.721782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:39.721782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:39.721782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:39.721782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:39.721782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:39.721782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:39.722648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:39.722648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:39.722648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:39.722648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:39.722648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:39.722648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:39.722648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:39.722648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:39.722648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:39.722648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:39.722648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:39.722648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:39.722648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:39.722648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:39.722648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:39.722648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:39.723485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:39.723485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:39.723485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:39.723485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:39.723485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:39.723485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:39.723485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:39.723485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:39.723485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:39.723485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:39.723485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:39.723485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:39.732297 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ad3a2-7f9a-42a2-b10e-a853a6ded3c4 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:39.732438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 179/344] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:39.743174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2ef32-ccb1-49fc-9fa6-b516c749d307 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:39.807134 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:39.914468 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2ef32-ccb1-49fc-9fa6-b516c749d307 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:39.965578 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b5fec2-d51c-4a02-bb25-f6304251b582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:39.966291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5fec2-d51c-4a02-bb25-f6304251b582 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:39.966774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5fec2-d51c-4a02-bb25-f6304251b582 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:39.967202 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5fec2-d51c-4a02-bb25-f6304251b582 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:39.968317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b5fec2-d51c-4a02-bb25-f6304251b582 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:39.968880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b5fec2-d51c-4a02-bb25-f6304251b582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:39.970949 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b5fec2-d51c-4a02-bb25-f6304251b582 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:39.980991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 180/345] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:39.982424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4367dc-e040-4b9d-b378-42554aa6f476 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:39.982895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4367dc-e040-4b9d-b378-42554aa6f476 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:39.983304 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4367dc-e040-4b9d-b378-42554aa6f476 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:39.984908 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 181/346] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:39.987572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ad3a2-7f9a-42a2-b10e-a853a6ded3c4 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:39.989134 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4367dc-e040-4b9d-b378-42554aa6f476 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:40.030349 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:40.043838 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773c49e-07fe-4447-a1f2-650fa4b0b9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:40.044759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773c49e-07fe-4447-a1f2-650fa4b0b9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.045193 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773c49e-07fe-4447-a1f2-650fa4b0b9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.045606 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773c49e-07fe-4447-a1f2-650fa4b0b9f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.046482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773c49e-07fe-4447-a1f2-650fa4b0b9f4 None admin] RBAC: Adding query filter params (group.id=f7bfc768836142d0b409afa8b8c12313, role.id=7d3cf58c1ad2407c9f8c60ba99b2006d, scope.project.id=886ef67601844188bb0aa19dcf32dea7) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:40.046983 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773c49e-07fe-4447-a1f2-650fa4b0b9f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:40.049139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773c49e-07fe-4447-a1f2-650fa4b0b9f4 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:40.062920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 166/347] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:39 2022] GET /identity/v3/role_assignments?group.id=f7bfc768836142d0b409afa8b8c12313&scope.project.id=886ef67601844188bb0aa19dcf32dea7&role.id=7d3cf58c1ad2407c9f8c60ba99b2006d => generated 260 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:40.064825 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e7242-b134-4781-84cd-d48ab230223d None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.065269 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e7242-b134-4781-84cd-d48ab230223d None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.065701 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e7242-b134-4781-84cd-d48ab230223d None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.067886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 167/348] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:40.069267 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fb817-1d57-429d-b35b-c05912c15046 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.069854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fb817-1d57-429d-b35b-c05912c15046 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.070354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4fb817-1d57-429d-b35b-c05912c15046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.096502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-9d4fb817-1d57-429d-b35b-c05912c15046 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:40.102907 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-9d4fb817-1d57-429d-b35b-c05912c15046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:40.105121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:40.128556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4367dc-e040-4b9d-b378-42554aa6f476 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:40.184418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a06d5-f409-49e9-b09f-929dbdb6eef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:40.185043 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a06d5-f409-49e9-b09f-929dbdb6eef8 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.185320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a06d5-f409-49e9-b09f-929dbdb6eef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.185587 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a06d5-f409-49e9-b09f-929dbdb6eef8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.196819 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 168/349] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:40.198660 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d7464-3322-4274-ada5-ad4df748ae11 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.199138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d7464-3322-4274-ada5-ad4df748ae11 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.199558 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d7464-3322-4274-ada5-ad4df748ae11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.232696 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-4d4d7464-3322-4274-ada5-ad4df748ae11 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:40.234182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4d4d7464-3322-4274-ada5-ad4df748ae11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:40.353600 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 169/350] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:40.401775 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a06d5-f409-49e9-b09f-929dbdb6eef8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:40.403297 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a06d5-f409-49e9-b09f-929dbdb6eef8 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:40.523069 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc209d-1930-4aa1-9ca3-f3195ca8ebf4 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.523364 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc209d-1930-4aa1-9ca3-f3195ca8ebf4 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.523580 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc209d-1930-4aa1-9ca3-f3195ca8ebf4 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.526333 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 170/351] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:40.531900 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:40.539544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b8996-6151-4b0d-b85e-a342ffa4a5f6 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.539972 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b8996-6151-4b0d-b85e-a342ffa4a5f6 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.540258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b8996-6151-4b0d-b85e-a342ffa4a5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.546502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 182/352] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:40.552710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a06d5-f409-49e9-b09f-929dbdb6eef8 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:40.562096 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-3f4b8996-6151-4b0d-b85e-a342ffa4a5f6 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:40.563424 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3f4b8996-6151-4b0d-b85e-a342ffa4a5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:40.685705 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:40.706529 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a06d5-f409-49e9-b09f-929dbdb6eef8 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:40.731265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 171/353] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:40.732509 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b8996-6151-4b0d-b85e-a342ffa4a5f6 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:40.777074 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e08f5-5cd8-49e4-a304-31828256cec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:40.777733 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e08f5-5cd8-49e4-a304-31828256cec6 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.777920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e08f5-5cd8-49e4-a304-31828256cec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.778188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e08f5-5cd8-49e4-a304-31828256cec6 None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.783932 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e08f5-5cd8-49e4-a304-31828256cec6 None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:40.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e08f5-5cd8-49e4-a304-31828256cec6 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:40.787665 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 183/354] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:40 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:40.794336 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e08f5-5cd8-49e4-a304-31828256cec6 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:40.891265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b8996-6151-4b0d-b85e-a342ffa4a5f6 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:40.893600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:40.942841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7f94f-9ecb-4d5f-a76d-243240b823f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:40.943068 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7f94f-9ecb-4d5f-a76d-243240b823f9 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:40.943290 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7f94f-9ecb-4d5f-a76d-243240b823f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:40.943543 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7f94f-9ecb-4d5f-a76d-243240b823f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:40.957010 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e08f5-5cd8-49e4-a304-31828256cec6 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:40.959509 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7f94f-9ecb-4d5f-a76d-243240b823f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:40.965313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-0aa7f94f-9ecb-4d5f-a76d-243240b823f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:40.965313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:40.965313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:40.965313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:40.965313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:40.965313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:40.965313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:40.965313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:40.965313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:40.965313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:40.965313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:40.965313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:40.965313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:40.965313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:40.965313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:40.965313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:40.966232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:40.966232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:40.966232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:40.966232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:40.966232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:40.966232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:40.966232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:40.966232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:40.966232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:40.966232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:40.966232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:40.966232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:40.966232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:40.966232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:40.966232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:40.966232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:40.967485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:40.967485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:40.967485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:40.967485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:40.967485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:40.967485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:40.967485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:40.967485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:40.967485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:40.967485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:40.967485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:40.967485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:40.967485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 172/355] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:40.968189 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7f94f-9ecb-4d5f-a76d-243240b823f9 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:41.008121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44a9b1-e115-455f-a0f5-4ca24da08bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:41.009140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44a9b1-e115-455f-a0f5-4ca24da08bcc None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.009627 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44a9b1-e115-455f-a0f5-4ca24da08bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.010099 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44a9b1-e115-455f-a0f5-4ca24da08bcc None admin] PATH_INFO: `/v3/groups/f7bfc768836142d0b409afa8b8c12313` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.012739 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f44a9b1-e115-455f-a0f5-4ca24da08bcc None admin] RBAC: Authorizing `identity:get_group(group_id=f7bfc768836142d0b409afa8b8c12313)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:41.014411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f44a9b1-e115-455f-a0f5-4ca24da08bcc None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:41.016780 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 184/356] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:40 2022] GET /identity/v3/groups/f7bfc768836142d0b409afa8b8c12313 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:41.019561 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0581541-f646-490c-9962-3db9139bc155 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.020117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0581541-f646-490c-9962-3db9139bc155 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.020582 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0581541-f646-490c-9962-3db9139bc155 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.022124 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 185/357] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:41.023209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0581541-f646-490c-9962-3db9139bc155 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:41.033411 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:41.108890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7f94f-9ecb-4d5f-a76d-243240b823f9 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:41.146852 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bf8bb-c33e-40e3-bd2d-20b1a86d1c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:41.147870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bf8bb-c33e-40e3-bd2d-20b1a86d1c3a None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.148365 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bf8bb-c33e-40e3-bd2d-20b1a86d1c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.148827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bf8bb-c33e-40e3-bd2d-20b1a86d1c3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.156056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0581541-f646-490c-9962-3db9139bc155 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:41.160436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bf8bb-c33e-40e3-bd2d-20b1a86d1c3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:41.165713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-127bf8bb-c33e-40e3-bd2d-20b1a86d1c3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:41.165713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:41.165713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:41.165713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:41.165713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:41.165713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:41.165713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:41.165713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:41.165713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:41.165713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:41.165713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:41.165713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:41.165713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:41.165713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:41.165713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:41.165713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:41.166605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:41.166605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:41.166605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:41.166605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:41.166605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:41.166605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:41.166605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:41.166605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:41.166605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:41.166605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:41.166605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:41.166605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:41.166605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:41.166605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:41.166605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:41.166605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:41.167468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:41.167468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:41.167468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:41.167468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:41.167468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:41.167468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:41.167468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:41.167468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:41.167468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:41.167468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:41.167468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:41.167468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:41.170065 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 173/358] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:41.171115 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bf8bb-c33e-40e3-bd2d-20b1a86d1c3a None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:41.197164 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:41.212470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884abb62-05dd-4ab7-a0a4-ca039d63eea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:41.214113 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884abb62-05dd-4ab7-a0a4-ca039d63eea3 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.214113 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884abb62-05dd-4ab7-a0a4-ca039d63eea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.214305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884abb62-05dd-4ab7-a0a4-ca039d63eea3 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.215371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884abb62-05dd-4ab7-a0a4-ca039d63eea3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:41.215829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884abb62-05dd-4ab7-a0a4-ca039d63eea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:41.218031 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884abb62-05dd-4ab7-a0a4-ca039d63eea3 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:41.234353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 186/359] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:41.236921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f9953-14ac-4bd1-a6f3-b6d819a0bd28 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.237354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f9953-14ac-4bd1-a6f3-b6d819a0bd28 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.237762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f9953-14ac-4bd1-a6f3-b6d819a0bd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.262651 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-b35f9953-14ac-4bd1-a6f3-b6d819a0bd28 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:41.264170 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b35f9953-14ac-4bd1-a6f3-b6d819a0bd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:41.288446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:41.290891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bf8bb-c33e-40e3-bd2d-20b1a86d1c3a None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:41.339234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626cace-054d-49f3-8af2-9c76a229a5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:41.339847 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626cace-054d-49f3-8af2-9c76a229a5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.340139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626cace-054d-49f3-8af2-9c76a229a5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.340391 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626cace-054d-49f3-8af2-9c76a229a5a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.341115 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0626cace-054d-49f3-8af2-9c76a229a5a2 None admin] RBAC: Adding query filter params (group.id=f7bfc768836142d0b409afa8b8c12313, role.id=1b632105fdba460b809a2ecd1d186409, scope.project.id=886ef67601844188bb0aa19dcf32dea7) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:41.341419 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0626cace-054d-49f3-8af2-9c76a229a5a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:41.344524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0626cace-054d-49f3-8af2-9c76a229a5a2 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:41.349828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 174/360] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:41 2022] GET /identity/v3/role_assignments?group.id=f7bfc768836142d0b409afa8b8c12313&scope.project.id=886ef67601844188bb0aa19dcf32dea7&role.id=1b632105fdba460b809a2ecd1d186409 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:41.350645 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626cace-054d-49f3-8af2-9c76a229a5a2 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:41.355792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 187/361] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:41.357018 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f9953-14ac-4bd1-a6f3-b6d819a0bd28 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:41.564444 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f9953-14ac-4bd1-a6f3-b6d819a0bd28 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:41.581344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626cace-054d-49f3-8af2-9c76a229a5a2 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:41.586562 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:41.613459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd749d-d048-41f4-b7a3-939607e48b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:41.614575 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd749d-d048-41f4-b7a3-939607e48b34 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.617208 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd749d-d048-41f4-b7a3-939607e48b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.617702 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd749d-d048-41f4-b7a3-939607e48b34 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.619417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd749d-d048-41f4-b7a3-939607e48b34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:41.620696 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd749d-d048-41f4-b7a3-939607e48b34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:41.622269 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd749d-d048-41f4-b7a3-939607e48b34 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:41.633326 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbb1b1-44a9-49e1-9cf5-809ae1420000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:41.634341 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbb1b1-44a9-49e1-9cf5-809ae1420000 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.634874 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbb1b1-44a9-49e1-9cf5-809ae1420000 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.635426 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbb1b1-44a9-49e1-9cf5-809ae1420000 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.641450 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 188/362] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:41.644333 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd749d-d048-41f4-b7a3-939607e48b34 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:41.659644 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cbb1b1-44a9-49e1-9cf5-809ae1420000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:41.670063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-07cbb1b1-44a9-49e1-9cf5-809ae1420000 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:41.670063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:41.670063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:41.670063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:41.670063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:41.670063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:41.670063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:41.670063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:41.670063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:41.670063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:41.670063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:41.670063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:41.670063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:41.670063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:41.670063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:41.670063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:41.671107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:41.671107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:41.671107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:41.671107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:41.671107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:41.671107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:41.671107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:41.671107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:41.671107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:41.671107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:41.671107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:41.671107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:41.671107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:41.671107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:41.671107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:41.671107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:41.671908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:41.671908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:41.671908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:41.671908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:41.671908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:41.671908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:41.671908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:41.671908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:41.671908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:41.671908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:41.671908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:41.671908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:41.674959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 175/363] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:44:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:41.677845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbb1b1-44a9-49e1-9cf5-809ae1420000 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:41.719170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:41.799080 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd749d-d048-41f4-b7a3-939607e48b34 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:41.818334 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbb1b1-44a9-49e1-9cf5-809ae1420000 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:41.842788 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890562f-3ea2-4c6a-959c-98faa789f6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:41.843335 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890562f-3ea2-4c6a-959c-98faa789f6da None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.843613 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890562f-3ea2-4c6a-959c-98faa789f6da None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.843886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890562f-3ea2-4c6a-959c-98faa789f6da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.859872 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd135df-e0d0-4b4b-9d6a-5b6fb2e9f2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:41.861481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd135df-e0d0-4b4b-9d6a-5b6fb2e9f2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:41.861962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd135df-e0d0-4b4b-9d6a-5b6fb2e9f2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:41.862436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd135df-e0d0-4b4b-9d6a-5b6fb2e9f2b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:41.906852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:41.934988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890562f-3ea2-4c6a-959c-98faa789f6da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:41.936471 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890562f-3ea2-4c6a-959c-98faa789f6da None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:41.964060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd135df-e0d0-4b4b-9d6a-5b6fb2e9f2b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:41.965825 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd135df-e0d0-4b4b-9d6a-5b6fb2e9f2b1 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:42.011112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:42.063425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 189/364] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:42.065179 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890562f-3ea2-4c6a-959c-98faa789f6da None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.104223 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 176/365] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:42.105028 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd135df-e0d0-4b4b-9d6a-5b6fb2e9f2b1 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.179582 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:42.204797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890562f-3ea2-4c6a-959c-98faa789f6da None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:42.232529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd135df-e0d0-4b4b-9d6a-5b6fb2e9f2b1 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:42.249588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3365622-e369-4a66-a8e7-afc95577781e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:42.250209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3365622-e369-4a66-a8e7-afc95577781e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:42.250484 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3365622-e369-4a66-a8e7-afc95577781e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:42.250770 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3365622-e369-4a66-a8e7-afc95577781e None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:42.251684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3365622-e369-4a66-a8e7-afc95577781e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:42.252538 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3365622-e369-4a66-a8e7-afc95577781e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:42.254687 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3365622-e369-4a66-a8e7-afc95577781e None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:42.265087 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 190/366] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:42.265930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3365622-e369-4a66-a8e7-afc95577781e None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.271170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:42.279674 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f8778-369b-4004-aa8a-7f1edb664f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:42.280389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f8778-369b-4004-aa8a-7f1edb664f29 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:42.280671 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f8778-369b-4004-aa8a-7f1edb664f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:42.280945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f8778-369b-4004-aa8a-7f1edb664f29 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:42.297099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f8778-369b-4004-aa8a-7f1edb664f29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:42.303423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-4c3f8778-369b-4004-aa8a-7f1edb664f29 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:42.303423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:42.303423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:42.303423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:42.303423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:42.303423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:42.303423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:42.303423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:42.303423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:42.303423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:42.303423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:42.303423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:42.303423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:42.303423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:42.303423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:42.303423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:42.304360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:42.304360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:42.304360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:42.304360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:42.304360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:42.304360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:42.304360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:42.304360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:42.304360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:42.304360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:42.304360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:42.304360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:42.304360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:42.304360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:42.304360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:42.304360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:42.305566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:42.305566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:42.305566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:42.305566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:42.305566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:42.305566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:42.305566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:42.305566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:42.305566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:42.305566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:42.305566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 14:44:42.305566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:42.305566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 177/367] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:42.310292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f8778-369b-4004-aa8a-7f1edb664f29 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.423464 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3365622-e369-4a66-a8e7-afc95577781e None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:42.441889 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f8778-369b-4004-aa8a-7f1edb664f29 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:42.476465 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04a9db-f959-498b-89f8-1c66051b4a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:42.477256 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04a9db-f959-498b-89f8-1c66051b4a60 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:42.477540 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04a9db-f959-498b-89f8-1c66051b4a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:42.477815 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04a9db-f959-498b-89f8-1c66051b4a60 None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:42.480153 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04a9db-f959-498b-89f8-1c66051b4a60 None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:42.481630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04a9db-f959-498b-89f8-1c66051b4a60 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:42.483673 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 191/368] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:42.484269 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04a9db-f959-498b-89f8-1c66051b4a60 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.499147 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:42.511393 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38369602-0e82-465c-be10-c7828fbbdbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:42.512144 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38369602-0e82-465c-be10-c7828fbbdbec None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:42.512512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38369602-0e82-465c-be10-c7828fbbdbec None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:42.512801 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38369602-0e82-465c-be10-c7828fbbdbec None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:42.514179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38369602-0e82-465c-be10-c7828fbbdbec None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:42.515751 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38369602-0e82-465c-be10-c7828fbbdbec None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:42.518472 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 178/369] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:42.520434 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38369602-0e82-465c-be10-c7828fbbdbec None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.587232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:42.605412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04a9db-f959-498b-89f8-1c66051b4a60 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:42.642261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031511a-9931-4de3-977d-048a34784f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:42.642628 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38369602-0e82-465c-be10-c7828fbbdbec None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:42.642950 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031511a-9931-4de3-977d-048a34784f23 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:42.643233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031511a-9931-4de3-977d-048a34784f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:42.643512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031511a-9931-4de3-977d-048a34784f23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:42.685292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2053028-41ab-4724-a9f7-26852ffcfa22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:42.685948 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2053028-41ab-4724-a9f7-26852ffcfa22 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:42.686227 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2053028-41ab-4724-a9f7-26852ffcfa22 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:42.686497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2053028-41ab-4724-a9f7-26852ffcfa22 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:42.687200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2053028-41ab-4724-a9f7-26852ffcfa22 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:42.687506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2053028-41ab-4724-a9f7-26852ffcfa22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:42.689075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2053028-41ab-4724-a9f7-26852ffcfa22 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:42.698604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 179/370] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:42.703693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2053028-41ab-4724-a9f7-26852ffcfa22 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.717091 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031511a-9931-4de3-977d-048a34784f23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:42.718684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d031511a-9931-4de3-977d-048a34784f23 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:42.735098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:42.817104 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2053028-41ab-4724-a9f7-26852ffcfa22 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:42.833037 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:42.833834 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 192/371] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:42.834759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031511a-9931-4de3-977d-048a34784f23 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.862168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b988b-92a1-497c-a8b7-abbadbacf138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:42.862859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b988b-92a1-497c-a8b7-abbadbacf138 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:42.863144 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b988b-92a1-497c-a8b7-abbadbacf138 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:42.863420 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b988b-92a1-497c-a8b7-abbadbacf138 None admin] PATH_INFO: `/v3/groups/f7bfc768836142d0b409afa8b8c12313` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:42.864558 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b988b-92a1-497c-a8b7-abbadbacf138 None admin] RBAC: Authorizing `identity:get_group(group_id=f7bfc768836142d0b409afa8b8c12313)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:42.866151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b988b-92a1-497c-a8b7-abbadbacf138 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:42.868115 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 180/372] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/groups/f7bfc768836142d0b409afa8b8c12313 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:42.868906 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b988b-92a1-497c-a8b7-abbadbacf138 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:42.965406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031511a-9931-4de3-977d-048a34784f23 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:42.977430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:42.988396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b988b-92a1-497c-a8b7-abbadbacf138 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:43.005009 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826a478-d2c3-492d-a35f-bbe4497fff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:43.005752 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826a478-d2c3-492d-a35f-bbe4497fff49 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:43.006035 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826a478-d2c3-492d-a35f-bbe4497fff49 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:43.006307 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826a478-d2c3-492d-a35f-bbe4497fff49 None admin] PATH_INFO: `/v3/groups/f7bfc768836142d0b409afa8b8c12313` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:43.007561 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826a478-d2c3-492d-a35f-bbe4497fff49 None admin] RBAC: Authorizing `identity:get_group(group_id=f7bfc768836142d0b409afa8b8c12313)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:43.009084 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826a478-d2c3-492d-a35f-bbe4497fff49 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:43.011123 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 193/373] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/groups/f7bfc768836142d0b409afa8b8c12313 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:43.011905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826a478-d2c3-492d-a35f-bbe4497fff49 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:43.025212 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe723473-08b0-4ae2-9b06-aff105da7c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:43.025855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe723473-08b0-4ae2-9b06-aff105da7c70 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:43.026136 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe723473-08b0-4ae2-9b06-aff105da7c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:43.026409 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe723473-08b0-4ae2-9b06-aff105da7c70 None admin] PATH_INFO: `/v3/users/24bfa515767042c382ce7147747a5eaa` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:43.027925 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe723473-08b0-4ae2-9b06-aff105da7c70 None admin] RBAC: Authorizing `identity:get_user(user_id=24bfa515767042c382ce7147747a5eaa)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:43.029376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe723473-08b0-4ae2-9b06-aff105da7c70 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:43.031022 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 181/374] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:44:42 2022] GET /identity/v3/users/24bfa515767042c382ce7147747a5eaa => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:43.031791 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe723473-08b0-4ae2-9b06-aff105da7c70 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:43.101178 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:43.133633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826a478-d2c3-492d-a35f-bbe4497fff49 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:43.173192 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe723473-08b0-4ae2-9b06-aff105da7c70 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:43.183273 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578037b-646d-4e1e-8b42-1d98880ec160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:43.183920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578037b-646d-4e1e-8b42-1d98880ec160 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:43.184192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578037b-646d-4e1e-8b42-1d98880ec160 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:43.184469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578037b-646d-4e1e-8b42-1d98880ec160 None admin] PATH_INFO: `/v3/projects/82e4fd5adbe54235a872491a4adc6897` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:43.185612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578037b-646d-4e1e-8b42-1d98880ec160 None admin] RBAC: Authorizing `identity:get_project(project_id=82e4fd5adbe54235a872491a4adc6897)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:43.187808 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578037b-646d-4e1e-8b42-1d98880ec160 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:43.188964 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 194/375] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:43 2022] GET /identity/v3/projects/82e4fd5adbe54235a872491a4adc6897 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:43.189572 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578037b-646d-4e1e-8b42-1d98880ec160 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:43.215620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a2e73-87ed-47a2-8c0b-96f703442781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:43.216620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a2e73-87ed-47a2-8c0b-96f703442781 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:43.217070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a2e73-87ed-47a2-8c0b-96f703442781 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:43.217524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a2e73-87ed-47a2-8c0b-96f703442781 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:43.218543 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a2e73-87ed-47a2-8c0b-96f703442781 None admin] RBAC: Adding query filter params (group.id=f7bfc768836142d0b409afa8b8c12313, role.id=7d3cf58c1ad2407c9f8c60ba99b2006d, scope.project.id=45e79ea1241346c58bf04c1494b3dc7d) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:43.219163 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a2e73-87ed-47a2-8c0b-96f703442781 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:43.221736 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a2e73-87ed-47a2-8c0b-96f703442781 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:43.229229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 182/376] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:43 2022] GET /identity/v3/role_assignments?group.id=f7bfc768836142d0b409afa8b8c12313&scope.project.id=45e79ea1241346c58bf04c1494b3dc7d&role.id=7d3cf58c1ad2407c9f8c60ba99b2006d => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:43.234800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a2e73-87ed-47a2-8c0b-96f703442781 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:43.255659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:43.314929 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578037b-646d-4e1e-8b42-1d98880ec160 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:43.358410 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78beaa5-514d-46a3-b914-ca2fed32501d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:43.359357 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78beaa5-514d-46a3-b914-ca2fed32501d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:43.359799 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78beaa5-514d-46a3-b914-ca2fed32501d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:43.360175 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78beaa5-514d-46a3-b914-ca2fed32501d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:43.361058 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78beaa5-514d-46a3-b914-ca2fed32501d None admin] RBAC: Adding query filter params (group.id=f7bfc768836142d0b409afa8b8c12313, role.id=1b632105fdba460b809a2ecd1d186409, scope.project.id=45e79ea1241346c58bf04c1494b3dc7d) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:43.361474 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78beaa5-514d-46a3-b914-ca2fed32501d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:43.362446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a2e73-87ed-47a2-8c0b-96f703442781 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:43.363249 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78beaa5-514d-46a3-b914-ca2fed32501d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:43.370455 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 195/377] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:43 2022] GET /identity/v3/role_assignments?group.id=f7bfc768836142d0b409afa8b8c12313&scope.project.id=45e79ea1241346c58bf04c1494b3dc7d&role.id=1b632105fdba460b809a2ecd1d186409 => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:43.372067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3fc68-a05c-40b3-9616-f0d114d4280b None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:43.372459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3fc68-a05c-40b3-9616-f0d114d4280b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:43.372834 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3fc68-a05c-40b3-9616-f0d114d4280b None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:43.374789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 196/378] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:43.375934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3fc68-a05c-40b3-9616-f0d114d4280b None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:43.379688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:43.421145 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240f194-cb7b-41f7-9ca9-1908143815cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:43.423652 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240f194-cb7b-41f7-9ca9-1908143815cf None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:43.423963 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240f194-cb7b-41f7-9ca9-1908143815cf None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:43.424237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240f194-cb7b-41f7-9ca9-1908143815cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:43.425690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240f194-cb7b-41f7-9ca9-1908143815cf None admin] RBAC: Adding query filter params (role.id=77e7991dcd664ba086355ef20ff81541, user.id=24bfa515767042c382ce7147747a5eaa, scope.system=all) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:43.426004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240f194-cb7b-41f7-9ca9-1908143815cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:43.428755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240f194-cb7b-41f7-9ca9-1908143815cf None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:43.460192 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 183/379] 10.222.0.26 () {56 vars in 1308 bytes} [Thu Sep 22 14:44:43 2022] GET /identity/v3/role_assignments?user.id=24bfa515767042c382ce7147747a5eaa&scope.system=all&role.id=77e7991dcd664ba086355ef20ff81541 => generated 510 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:43.464903 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd051d2-563d-4397-9580-a78fbd3a8832 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:43.465221 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd051d2-563d-4397-9580-a78fbd3a8832 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:43.465463 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd051d2-563d-4397-9580-a78fbd3a8832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:43.498051 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-cdd051d2-563d-4397-9580-a78fbd3a8832 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:43.499214 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-cdd051d2-563d-4397-9580-a78fbd3a8832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:43.537559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3fc68-a05c-40b3-9616-f0d114d4280b None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:43.552966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:43.602097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a823ac0-83b0-4180-b209-561c0e0e1894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:43.603156 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a823ac0-83b0-4180-b209-561c0e0e1894 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:43.603657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a823ac0-83b0-4180-b209-561c0e0e1894 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:43.606045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a823ac0-83b0-4180-b209-561c0e0e1894 None admin] PATH_INFO: `/v3/groups/abad01993c6c45c3b199eb371488b694` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:43.619653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a823ac0-83b0-4180-b209-561c0e0e1894 None admin] RBAC: Authorizing `identity:get_group(group_id=abad01993c6c45c3b199eb371488b694)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:43.623508 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a823ac0-83b0-4180-b209-561c0e0e1894 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:43.628787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 197/380] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:43 2022] GET /identity/v3/groups/abad01993c6c45c3b199eb371488b694 => generated 229 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:43.633453 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a823ac0-83b0-4180-b209-561c0e0e1894 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:43.670906 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 184/381] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:43.761923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a823ac0-83b0-4180-b209-561c0e0e1894 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:43.803191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cccab9-ca05-412e-8b65-e1db9eb42ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:43.803922 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cccab9-ca05-412e-8b65-e1db9eb42ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:43.804210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cccab9-ca05-412e-8b65-e1db9eb42ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:43.804482 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cccab9-ca05-412e-8b65-e1db9eb42ba3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:43.805221 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cccab9-ca05-412e-8b65-e1db9eb42ba3 None admin] RBAC: Adding query filter params (group.id=abad01993c6c45c3b199eb371488b694, role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=82e4fd5adbe54235a872491a4adc6897) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:43.805525 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cccab9-ca05-412e-8b65-e1db9eb42ba3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:43.807883 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cccab9-ca05-412e-8b65-e1db9eb42ba3 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:43.814760 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 198/382] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:43 2022] GET /identity/v3/role_assignments?group.id=abad01993c6c45c3b199eb371488b694&scope.project.id=82e4fd5adbe54235a872491a4adc6897&role.id=32c8a81e339a462e973e76bb528ef35a => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:43.904682 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206060a8-63ab-4f70-b0b8-ea90323a6d70 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:43.905306 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206060a8-63ab-4f70-b0b8-ea90323a6d70 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:43.905781 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206060a8-63ab-4f70-b0b8-ea90323a6d70 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:43.907165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 185/383] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:43.913591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb412d-eeb4-409f-a6d0-41035d2b1798 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:43.914074 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb412d-eeb4-409f-a6d0-41035d2b1798 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:43.914423 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb412d-eeb4-409f-a6d0-41035d2b1798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:43.941009 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-9fbb412d-eeb4-409f-a6d0-41035d2b1798 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:43.942579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-9fbb412d-eeb4-409f-a6d0-41035d2b1798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:43.980965 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:44.024328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 199/384] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:44.066462 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206060a8-63ab-4f70-b0b8-ea90323a6d70 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:44.201598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206060a8-63ab-4f70-b0b8-ea90323a6d70 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:44.204259 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:44.250360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6269950-adf2-4f85-aab9-8378c9d8d4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:44.251804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269950-adf2-4f85-aab9-8378c9d8d4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:44.252532 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269950-adf2-4f85-aab9-8378c9d8d4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:44.253174 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269950-adf2-4f85-aab9-8378c9d8d4f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:44.269960 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6269950-adf2-4f85-aab9-8378c9d8d4f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:44.276166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-a6269950-adf2-4f85-aab9-8378c9d8d4f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:44.276166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:44.276166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:44.276166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:44.276166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:44.276166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:44.276166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:44.276166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:44.276166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:44.276166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:44.276166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:44.276166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:44.276166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:44.276166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:44.276166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:44.276166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:44.277101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:44.277101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:44.277101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:44.277101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:44.277101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:44.277101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:44.277101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:44.277101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:44.277101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:44.277101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:44.277101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:44.277101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:44.277101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:44.277101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:44.277101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:44.277101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:44.277901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:44.277901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:44.277901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:44.277901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:44.277901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:44.277901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:44.277901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:44.277901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:44.277901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:44.277901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:44.277901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:44.277901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:44.280566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 186/385] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:44.291031 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb412d-eeb4-409f-a6d0-41035d2b1798 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:44.346669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6269950-adf2-4f85-aab9-8378c9d8d4f9 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:44.487472 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb412d-eeb4-409f-a6d0-41035d2b1798 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:44.526713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6269950-adf2-4f85-aab9-8378c9d8d4f9 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:44.552081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7c99b-b701-4169-a746-ef7b68fdc26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:44.552701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7c99b-b701-4169-a746-ef7b68fdc26e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:44.552983 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7c99b-b701-4169-a746-ef7b68fdc26e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:44.553255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7c99b-b701-4169-a746-ef7b68fdc26e None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:44.553989 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7c99b-b701-4169-a746-ef7b68fdc26e None admin] RBAC: Adding query filter params (name=member) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:44.554292 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7c99b-b701-4169-a746-ef7b68fdc26e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:44.555773 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7c99b-b701-4169-a746-ef7b68fdc26e None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:44.568148 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 200/386] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:44.580289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7c99b-b701-4169-a746-ef7b68fdc26e None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:44.597614 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d411c98-30fa-49a1-9102-234d3a102c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:44.599723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d411c98-30fa-49a1-9102-234d3a102c7b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:44.600423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d411c98-30fa-49a1-9102-234d3a102c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:44.601627 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d411c98-30fa-49a1-9102-234d3a102c7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:44.622598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d411c98-30fa-49a1-9102-234d3a102c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:44.630898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-7d411c98-30fa-49a1-9102-234d3a102c7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:44.630898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:44.630898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:44.630898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:44.630898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:44.630898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:44.630898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:44.630898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:44.630898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:44.630898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:44.630898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:44.630898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:44.630898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:44.630898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:44.630898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:44.630898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:44.632024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:44.632024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:44.632024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:44.632024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:44.632024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:44.632024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:44.632024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:44.632024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:44.632024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:44.632024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:44.632024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:44.632024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:44.632024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:44.632024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:44.632024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:44.632024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:44.632994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:44.632994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:44.632994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:44.632994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:44.632994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:44.632994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:44.632994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:44.632994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:44.632994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:44.632994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:44.632994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:44.632994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:44.633848 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:44.646839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 187/387] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:44.655564 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d411c98-30fa-49a1-9102-234d3a102c7b None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:44.708531 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:44.752962 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7c99b-b701-4169-a746-ef7b68fdc26e None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:44.808960 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e1fe4-5d7f-438c-b80e-c217e93c7386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:44.809825 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e1fe4-5d7f-438c-b80e-c217e93c7386 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:44.815126 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e1fe4-5d7f-438c-b80e-c217e93c7386 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:44.815413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e1fe4-5d7f-438c-b80e-c217e93c7386 None admin] PATH_INFO: `/v3/groups/f7bfc768836142d0b409afa8b8c12313` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:44.816647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e1fe4-5d7f-438c-b80e-c217e93c7386 None admin] RBAC: Authorizing `identity:get_group(group_id=f7bfc768836142d0b409afa8b8c12313)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:44.818099 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e1fe4-5d7f-438c-b80e-c217e93c7386 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:44.822756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 201/388] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:44 2022] GET /identity/v3/groups/f7bfc768836142d0b409afa8b8c12313 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:44.835751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e1fe4-5d7f-438c-b80e-c217e93c7386 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:44.859495 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d411c98-30fa-49a1-9102-234d3a102c7b None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:44.923094 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7ae32-c83b-4406-9fd5-0c4e603caeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:44.924556 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7ae32-c83b-4406-9fd5-0c4e603caeec None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:44.924976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7ae32-c83b-4406-9fd5-0c4e603caeec None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:44.925384 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7ae32-c83b-4406-9fd5-0c4e603caeec None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:44.926229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7ae32-c83b-4406-9fd5-0c4e603caeec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:44.926671 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7ae32-c83b-4406-9fd5-0c4e603caeec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:44.929693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7ae32-c83b-4406-9fd5-0c4e603caeec None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:44.947361 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 188/389] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:44.953341 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7ae32-c83b-4406-9fd5-0c4e603caeec None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:45.039644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e1fe4-5d7f-438c-b80e-c217e93c7386 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:45.046967 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:45.112900 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f4f4d-582a-48c0-b98f-083b403ee14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:45.113797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f4f4d-582a-48c0-b98f-083b403ee14a None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:45.114241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f4f4d-582a-48c0-b98f-083b403ee14a None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:45.114682 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f4f4d-582a-48c0-b98f-083b403ee14a None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:45.116241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f4f4d-582a-48c0-b98f-083b403ee14a None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:45.117824 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f4f4d-582a-48c0-b98f-083b403ee14a None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:45.121099 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 202/390] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:44 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:45.128522 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:45.138162 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f4f4d-582a-48c0-b98f-083b403ee14a None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:45.162395 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7ae32-c83b-4406-9fd5-0c4e603caeec None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:45.224594 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea57348-f17b-482f-81ab-ea222417543f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:45.225902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea57348-f17b-482f-81ab-ea222417543f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:45.226328 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea57348-f17b-482f-81ab-ea222417543f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:45.226790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea57348-f17b-482f-81ab-ea222417543f None admin] PATH_INFO: `/v3/groups/f7bfc768836142d0b409afa8b8c12313` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:45.228210 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea57348-f17b-482f-81ab-ea222417543f None admin] RBAC: Authorizing `identity:get_group(group_id=f7bfc768836142d0b409afa8b8c12313)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:45.229857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea57348-f17b-482f-81ab-ea222417543f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:45.231973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 189/391] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:44 2022] GET /identity/v3/groups/f7bfc768836142d0b409afa8b8c12313 => generated 226 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:45.237867 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea57348-f17b-482f-81ab-ea222417543f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:45.331013 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f4f4d-582a-48c0-b98f-083b403ee14a None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:45.396000 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c808d-86c5-4922-833d-1b6fce392db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:45.396683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c808d-86c5-4922-833d-1b6fce392db7 None admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:45.396944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c808d-86c5-4922-833d-1b6fce392db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:45.397218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c808d-86c5-4922-833d-1b6fce392db7 None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7/groups/f7bfc768836142d0b409afa8b8c12313/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:45.399696 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c808d-86c5-4922-833d-1b6fce392db7 None admin] RBAC: Authorizing `identity:create_grant(project_id=886ef67601844188bb0aa19dcf32dea7, group_id=f7bfc768836142d0b409afa8b8c12313, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:45.402051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c808d-86c5-4922-833d-1b6fce392db7 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:45.411048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea57348-f17b-482f-81ab-ea222417543f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:45.425641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 203/392] 10.222.0.26 () {58 vars in 1442 bytes} [Thu Sep 22 14:44:45 2022] PUT /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7/groups/f7bfc768836142d0b409afa8b8c12313/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:44:45.462266 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacba0d-38e9-41ac-896f-4c4c2e579d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:45.462957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacba0d-38e9-41ac-896f-4c4c2e579d4f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:45.463240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacba0d-38e9-41ac-896f-4c4c2e579d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:45.463511 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacba0d-38e9-41ac-896f-4c4c2e579d4f None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:45.464681 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacba0d-38e9-41ac-896f-4c4c2e579d4f None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:45.466310 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacba0d-38e9-41ac-896f-4c4c2e579d4f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:45.468407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 190/393] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:45 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:45.475577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c808d-86c5-4922-833d-1b6fce392db7 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:45.507049 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:45.615365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c808d-86c5-4922-833d-1b6fce392db7 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:45.659585 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27798d0b-f8e9-4fbe-8cef-2a3c599bdacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:45.660193 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27798d0b-f8e9-4fbe-8cef-2a3c599bdacb None admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:45.660475 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27798d0b-f8e9-4fbe-8cef-2a3c599bdacb None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:45.660757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27798d0b-f8e9-4fbe-8cef-2a3c599bdacb None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7/groups/f7bfc768836142d0b409afa8b8c12313/roles/1b632105fdba460b809a2ecd1d186409` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:45.664567 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27798d0b-f8e9-4fbe-8cef-2a3c599bdacb None admin] RBAC: Authorizing `identity:create_grant(project_id=886ef67601844188bb0aa19dcf32dea7, group_id=f7bfc768836142d0b409afa8b8c12313, role_id=1b632105fdba460b809a2ecd1d186409)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:45.666119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27798d0b-f8e9-4fbe-8cef-2a3c599bdacb None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:45.692075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 204/394] 10.222.0.26 () {58 vars in 1442 bytes} [Thu Sep 22 14:44:45 2022] PUT /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7/groups/f7bfc768836142d0b409afa8b8c12313/roles/1b632105fdba460b809a2ecd1d186409 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:46.326203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74edcc-8927-4bf2-a9f5-53eb801c20de None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:46.326753 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74edcc-8927-4bf2-a9f5-53eb801c20de None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:46.326753 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74edcc-8927-4bf2-a9f5-53eb801c20de None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:46.331673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 191/395] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:46.339790 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd75f2-356c-4b3f-96e0-5b7a3c9d6056 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:46.340679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd75f2-356c-4b3f-96e0-5b7a3c9d6056 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:46.341157 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd75f2-356c-4b3f-96e0-5b7a3c9d6056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:46.364965 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-12cd75f2-356c-4b3f-96e0-5b7a3c9d6056 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:46.366521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-12cd75f2-356c-4b3f-96e0-5b7a3c9d6056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:46.377499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc51792-10d5-45d4-baa0-babe85f9e45e None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:46.378107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc51792-10d5-45d4-baa0-babe85f9e45e None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:46.378581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc51792-10d5-45d4-baa0-babe85f9e45e None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:46.380140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 192/396] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:46.385483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06b9d0-3906-463f-986d-850ac54748a9 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:46.385764 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06b9d0-3906-463f-986d-850ac54748a9 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:46.386034 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06b9d0-3906-463f-986d-850ac54748a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:46.410820 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-cc06b9d0-3906-463f-986d-850ac54748a9 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:46.412592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-cc06b9d0-3906-463f-986d-850ac54748a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:46.430096 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:46.454264 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 205/397] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:46.498337 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 193/398] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:46.770144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd75f2-356c-4b3f-96e0-5b7a3c9d6056 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:46.794692 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:46.801941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06b9d0-3906-463f-986d-850ac54748a9 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:46.888126 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd75f2-356c-4b3f-96e0-5b7a3c9d6056 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:46.925777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c7efb-4cb8-4f88-9463-d2b155a84d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:46.926425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c7efb-4cb8-4f88-9463-d2b155a84d50 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:46.926704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c7efb-4cb8-4f88-9463-d2b155a84d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:46.927004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c7efb-4cb8-4f88-9463-d2b155a84d50 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:46.941185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c7efb-4cb8-4f88-9463-d2b155a84d50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:46.947142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-094c7efb-4cb8-4f88-9463-d2b155a84d50 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:46.947142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:46.947142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:46.947142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:46.947142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:46.947142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:46.947142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:46.947142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:46.947142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:46.947142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:46.947142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:46.947142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:46.947142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:46.947142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:46.947142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:46.947142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:46.948038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:46.948038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:46.948038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:46.948038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:46.948038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:46.948038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:46.948038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:46.948038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:46.948038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:46.948038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:46.948038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:46.948038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:46.948038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:46.948038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:46.948038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:46.948038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:46.948831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:46.948831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:46.948831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:46.948831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:46.948831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:46.948831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:46.948831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:46.948831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:46.948831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:46.948831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:46.948831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:46.948831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:46.948831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 206/399] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:46.957264 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c7efb-4cb8-4f88-9463-d2b155a84d50 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.012769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06b9d0-3906-463f-986d-850ac54748a9 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:47.063203 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:47.086861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c7efb-4cb8-4f88-9463-d2b155a84d50 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:47.127219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1e423-2731-4613-b7d9-6c2977eefa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:47.128263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1e423-2731-4613-b7d9-6c2977eefa7d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.128748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1e423-2731-4613-b7d9-6c2977eefa7d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.128933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce251a4a-5873-4b09-86cf-aa452cf55a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:47.129458 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1e423-2731-4613-b7d9-6c2977eefa7d None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.129561 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce251a4a-5873-4b09-86cf-aa452cf55a3c None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.129825 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce251a4a-5873-4b09-86cf-aa452cf55a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.130097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce251a4a-5873-4b09-86cf-aa452cf55a3c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.130977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1e423-2731-4613-b7d9-6c2977eefa7d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:47.131501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1e423-2731-4613-b7d9-6c2977eefa7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:47.133158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1e423-2731-4613-b7d9-6c2977eefa7d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:47.143329 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 207/400] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:47.149114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce251a4a-5873-4b09-86cf-aa452cf55a3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:47.149403 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe72eb-5abc-4855-b802-be5f9817721e None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.149928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe72eb-5abc-4855-b802-be5f9817721e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.150040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-ce251a4a-5873-4b09-86cf-aa452cf55a3c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:47.150040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:47.150040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:47.150040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:47.150040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:47.150040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:47.150040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:47.150040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:47.150040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:47.150040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:47.150040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:47.150040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:47.150040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:47.150040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:47.150040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:47.150040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:47.150981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:47.150981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:47.150981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:47.150981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:47.150981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:47.150981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:47.150981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:47.150981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:47.150981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:47.150981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:47.150981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:47.150981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:47.150981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:47.150981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:47.150981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:47.150981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:47.151779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:47.151779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:47.151779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:47.151779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:47.151779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:47.151779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:47.151779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:47.151779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:47.151779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:47.151779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:47.151779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:47.151779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:47.151779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 194/401] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:47.152438 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce251a4a-5873-4b09-86cf-aa452cf55a3c None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.154450 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe72eb-5abc-4855-b802-be5f9817721e None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.156534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 208/402] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:47 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:47.157574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe72eb-5abc-4855-b802-be5f9817721e None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.169027 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:47.267853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce251a4a-5873-4b09-86cf-aa452cf55a3c None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:47.282427 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe72eb-5abc-4855-b802-be5f9817721e None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:47.310174 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e10304-8e9f-4fda-868a-175e2cf5f832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:47.311150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e10304-8e9f-4fda-868a-175e2cf5f832 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.311150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e10304-8e9f-4fda-868a-175e2cf5f832 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.311497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e10304-8e9f-4fda-868a-175e2cf5f832 None admin] PATH_INFO: `/v3/groups/f7bfc768836142d0b409afa8b8c12313` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.312709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e10304-8e9f-4fda-868a-175e2cf5f832 None admin] RBAC: Authorizing `identity:get_group(group_id=f7bfc768836142d0b409afa8b8c12313)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:47.314765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e10304-8e9f-4fda-868a-175e2cf5f832 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:47.321353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 195/403] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:47 2022] GET /identity/v3/groups/f7bfc768836142d0b409afa8b8c12313 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:47.321353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea72801-28b0-4fcf-bc31-1edf79f987c8 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.321353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea72801-28b0-4fcf-bc31-1edf79f987c8 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.321353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea72801-28b0-4fcf-bc31-1edf79f987c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.338810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e3acb-8572-427c-b708-89defa073e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:47.339570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e3acb-8572-427c-b708-89defa073e60 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.339856 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e3acb-8572-427c-b708-89defa073e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.340384 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e3acb-8572-427c-b708-89defa073e60 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.340903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e3acb-8572-427c-b708-89defa073e60 None admin] RBAC: Adding query filter params (name=member) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:47.341210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e3acb-8572-427c-b708-89defa073e60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:47.342695 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e3acb-8572-427c-b708-89defa073e60 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:47.347144 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-dea72801-28b0-4fcf-bc31-1edf79f987c8 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:47.348853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-dea72801-28b0-4fcf-bc31-1edf79f987c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:47.349180 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:47.362394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 209/404] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:47.362394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e3acb-8572-427c-b708-89defa073e60 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.435685 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 196/405] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:47.436785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea72801-28b0-4fcf-bc31-1edf79f987c8 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.453133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:47.510655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e3acb-8572-427c-b708-89defa073e60 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:47.588937 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea72801-28b0-4fcf-bc31-1edf79f987c8 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:47.591102 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd8974-fd9e-4ad9-98b0-65dc55f5d446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:47.591746 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd8974-fd9e-4ad9-98b0-65dc55f5d446 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.592020 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd8974-fd9e-4ad9-98b0-65dc55f5d446 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.592287 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd8974-fd9e-4ad9-98b0-65dc55f5d446 None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.593456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd8974-fd9e-4ad9-98b0-65dc55f5d446 None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:47.594932 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd8974-fd9e-4ad9-98b0-65dc55f5d446 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:47.604431 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 210/406] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:47 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:47.605309 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd8974-fd9e-4ad9-98b0-65dc55f5d446 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.665138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a177010-cad2-4eec-ac93-755f04b9b91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:47.666251 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a177010-cad2-4eec-ac93-755f04b9b91d None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.666918 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a177010-cad2-4eec-ac93-755f04b9b91d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.668269 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a177010-cad2-4eec-ac93-755f04b9b91d None admin] PATH_INFO: `/v3/groups/f7bfc768836142d0b409afa8b8c12313` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.671581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a177010-cad2-4eec-ac93-755f04b9b91d None admin] RBAC: Authorizing `identity:get_group(group_id=f7bfc768836142d0b409afa8b8c12313)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:47.673350 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a177010-cad2-4eec-ac93-755f04b9b91d None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:47.683016 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 197/407] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:47 2022] GET /identity/v3/groups/f7bfc768836142d0b409afa8b8c12313 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:47.689970 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a177010-cad2-4eec-ac93-755f04b9b91d None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.733089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:47.783379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd8974-fd9e-4ad9-98b0-65dc55f5d446 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:47.809644 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:47.824656 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138886b-80d5-4a03-b1b5-61b9fbf65d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:47.825479 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138886b-80d5-4a03-b1b5-61b9fbf65d65 None admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.825869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138886b-80d5-4a03-b1b5-61b9fbf65d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.826255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138886b-80d5-4a03-b1b5-61b9fbf65d65 None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d/groups/f7bfc768836142d0b409afa8b8c12313/roles/1b632105fdba460b809a2ecd1d186409` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.829303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138886b-80d5-4a03-b1b5-61b9fbf65d65 None admin] RBAC: Authorizing `identity:create_grant(project_id=45e79ea1241346c58bf04c1494b3dc7d, group_id=f7bfc768836142d0b409afa8b8c12313, role_id=1b632105fdba460b809a2ecd1d186409)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:47.830844 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138886b-80d5-4a03-b1b5-61b9fbf65d65 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:47.833412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a177010-cad2-4eec-ac93-755f04b9b91d None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:47.848796 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 211/408] 10.222.0.26 () {58 vars in 1442 bytes} [Thu Sep 22 14:44:47 2022] PUT /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d/groups/f7bfc768836142d0b409afa8b8c12313/roles/1b632105fdba460b809a2ecd1d186409 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:47.849911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138886b-80d5-4a03-b1b5-61b9fbf65d65 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.877253 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449f1ff-a982-4b49-86b4-56babed54011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:47.878146 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449f1ff-a982-4b49-86b4-56babed54011 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:47.878535 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449f1ff-a982-4b49-86b4-56babed54011 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:47.878948 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a449f1ff-a982-4b49-86b4-56babed54011 None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:47.880623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449f1ff-a982-4b49-86b4-56babed54011 None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:47.882379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a449f1ff-a982-4b49-86b4-56babed54011 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:47.884831 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 198/409] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:47 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:47.894771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449f1ff-a982-4b49-86b4-56babed54011 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:47.978287 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138886b-80d5-4a03-b1b5-61b9fbf65d65 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:48.010325 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:48.019743 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324ba2b-ba9c-4ed6-b393-af87ca356e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:48.020530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324ba2b-ba9c-4ed6-b393-af87ca356e0d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:48.021191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324ba2b-ba9c-4ed6-b393-af87ca356e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:48.021580 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324ba2b-ba9c-4ed6-b393-af87ca356e0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:48.034943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324ba2b-ba9c-4ed6-b393-af87ca356e0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:48.041797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-e324ba2b-ba9c-4ed6-b393-af87ca356e0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:48.041797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:48.041797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:48.041797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:48.041797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:48.041797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:48.041797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:48.041797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:48.041797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:48.041797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:48.041797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:48.041797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:48.041797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:48.041797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:48.041797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:48.041797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:48.043028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:48.043028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:48.043028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:48.043028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:48.043028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:48.043028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:48.043028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:48.043028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:48.043028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:48.043028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:48.043028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:48.043028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:48.043028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:48.043028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:48.043028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:48.043028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:48.043990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:48.043990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:48.043990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:48.043990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:48.043990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:48.043990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:48.043990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:48.043990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:48.043990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:48.043990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:48.043990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:48.043990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:48.045999 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 212/410] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:44:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:48.055419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324ba2b-ba9c-4ed6-b393-af87ca356e0d None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:48.083056 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a449f1ff-a982-4b49-86b4-56babed54011 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:48.161813 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f36de6-785d-44f6-acd4-5f1eeeb9188d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:48.162419 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f36de6-785d-44f6-acd4-5f1eeeb9188d None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:48.162694 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f36de6-785d-44f6-acd4-5f1eeeb9188d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:48.162989 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f36de6-785d-44f6-acd4-5f1eeeb9188d None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d/groups/f7bfc768836142d0b409afa8b8c12313/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:48.168043 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f36de6-785d-44f6-acd4-5f1eeeb9188d None admin] RBAC: Authorizing `identity:create_grant(project_id=45e79ea1241346c58bf04c1494b3dc7d, group_id=f7bfc768836142d0b409afa8b8c12313, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:48.171746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f36de6-785d-44f6-acd4-5f1eeeb9188d None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:48.193603 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 199/411] 10.222.0.26 () {58 vars in 1442 bytes} [Thu Sep 22 14:44:47 2022] PUT /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d/groups/f7bfc768836142d0b409afa8b8c12313/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:44:48.234043 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e61fb-97a8-4239-a41c-d24af04f09b7 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:48.234434 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e61fb-97a8-4239-a41c-d24af04f09b7 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:48.234819 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e61fb-97a8-4239-a41c-d24af04f09b7 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:48.236688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 200/412] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:48.242338 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd069f-9baa-46c1-a6e4-5fda3d2e1d93 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:48.242782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd069f-9baa-46c1-a6e4-5fda3d2e1d93 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:48.243043 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd069f-9baa-46c1-a6e4-5fda3d2e1d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:48.255458 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324ba2b-ba9c-4ed6-b393-af87ca356e0d None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:48.272213 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:48.276428 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-0ffd069f-9baa-46c1-a6e4-5fda3d2e1d93 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:48.279994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0ffd069f-9baa-46c1-a6e4-5fda3d2e1d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:48.334789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9546d1-1cbe-4d26-8519-e20b6df1b0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:48.334789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9546d1-1cbe-4d26-8519-e20b6df1b0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:48.334789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9546d1-1cbe-4d26-8519-e20b6df1b0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:48.334789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9546d1-1cbe-4d26-8519-e20b6df1b0c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:48.335459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9546d1-1cbe-4d26-8519-e20b6df1b0c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:48.335459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9546d1-1cbe-4d26-8519-e20b6df1b0c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:48.335459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9546d1-1cbe-4d26-8519-e20b6df1b0c0 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:48.361059 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 213/413] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:48.362051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9546d1-1cbe-4d26-8519-e20b6df1b0c0 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:48.432432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 201/414] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:48.472486 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:48.529062 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9546d1-1cbe-4d26-8519-e20b6df1b0c0 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:48.559849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25410f9c-8ce9-409f-a6a0-07dbf16bf3c3 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:48.560163 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25410f9c-8ce9-409f-a6a0-07dbf16bf3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:48.560420 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25410f9c-8ce9-409f-a6a0-07dbf16bf3c3 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:48.562103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 202/415] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:48.567510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee79016-9d10-4f2f-8a2c-fabbed89b2e8 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:48.567903 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee79016-9d10-4f2f-8a2c-fabbed89b2e8 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:48.568286 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee79016-9d10-4f2f-8a2c-fabbed89b2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:48.569225 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3381d94-c981-4d78-8306-c6e3e42708c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:48.569868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3381d94-c981-4d78-8306-c6e3e42708c7 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:48.570143 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3381d94-c981-4d78-8306-c6e3e42708c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:48.570412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3381d94-c981-4d78-8306-c6e3e42708c7 None admin] PATH_INFO: `/v3/groups/abad01993c6c45c3b199eb371488b694` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:48.571563 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3381d94-c981-4d78-8306-c6e3e42708c7 None admin] RBAC: Authorizing `identity:get_group(group_id=abad01993c6c45c3b199eb371488b694)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:48.573011 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3381d94-c981-4d78-8306-c6e3e42708c7 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:48.574905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 214/416] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:48 2022] GET /identity/v3/groups/abad01993c6c45c3b199eb371488b694 => generated 229 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:48.580596 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3381d94-c981-4d78-8306-c6e3e42708c7 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:48.601610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-fee79016-9d10-4f2f-8a2c-fabbed89b2e8 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:48.602827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-fee79016-9d10-4f2f-8a2c-fabbed89b2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:48.738973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 203/417] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:48.739864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee79016-9d10-4f2f-8a2c-fabbed89b2e8 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:48.753973 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3381d94-c981-4d78-8306-c6e3e42708c7 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:48.780951 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:48.818485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dcc80-2c60-4407-9e26-7e429183f05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:48.819160 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dcc80-2c60-4407-9e26-7e429183f05f None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:48.819436 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dcc80-2c60-4407-9e26-7e429183f05f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:48.819704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dcc80-2c60-4407-9e26-7e429183f05f None admin] PATH_INFO: `/v3/projects/82e4fd5adbe54235a872491a4adc6897` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:48.822138 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dcc80-2c60-4407-9e26-7e429183f05f None admin] RBAC: Authorizing `identity:get_project(project_id=82e4fd5adbe54235a872491a4adc6897)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:48.823597 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dcc80-2c60-4407-9e26-7e429183f05f None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:48.830261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 215/418] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:48 2022] GET /identity/v3/projects/82e4fd5adbe54235a872491a4adc6897 => generated 344 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:48.832567 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dcc80-2c60-4407-9e26-7e429183f05f None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:48.856755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:48.928985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee79016-9d10-4f2f-8a2c-fabbed89b2e8 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:48.946476 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dcc80-2c60-4407-9e26-7e429183f05f None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:48.984329 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c9cd7-a435-44c2-8f4f-a331ae4caf29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:48.984999 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c9cd7-a435-44c2-8f4f-a331ae4caf29 None admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:48.985276 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c9cd7-a435-44c2-8f4f-a331ae4caf29 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:48.985545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c9cd7-a435-44c2-8f4f-a331ae4caf29 None admin] PATH_INFO: `/v3/projects/82e4fd5adbe54235a872491a4adc6897/groups/abad01993c6c45c3b199eb371488b694/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:48.988856 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c9cd7-a435-44c2-8f4f-a331ae4caf29 None admin] RBAC: Authorizing `identity:create_grant(project_id=82e4fd5adbe54235a872491a4adc6897, group_id=abad01993c6c45c3b199eb371488b694, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:48.989777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c9cd7-a435-44c2-8f4f-a331ae4caf29 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:49.005824 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 216/419] 10.222.0.26 () {58 vars in 1442 bytes} [Thu Sep 22 14:44:48 2022] PUT /identity/v3/projects/82e4fd5adbe54235a872491a4adc6897/groups/abad01993c6c45c3b199eb371488b694/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:44:49.006770 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7c76e-01ff-4d1d-ba33-c83a09af28f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:49.007586 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7c76e-01ff-4d1d-ba33-c83a09af28f3 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:49.007993 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7c76e-01ff-4d1d-ba33-c83a09af28f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:49.008419 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed7c76e-01ff-4d1d-ba33-c83a09af28f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:49.021751 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed7c76e-01ff-4d1d-ba33-c83a09af28f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:49.027260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-4ed7c76e-01ff-4d1d-ba33-c83a09af28f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:49.027260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:49.027260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:49.027260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:49.027260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:49.027260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:49.027260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:49.027260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:49.027260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:49.027260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:49.027260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:49.027260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:49.027260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:49.027260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:49.027260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:49.027260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:49.028249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:49.028249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:49.028249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:49.028249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:49.028249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:49.028249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:49.028249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:49.028249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:49.028249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:49.028249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:49.028249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:49.028249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:49.028249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:49.028249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:49.028249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:49.028249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:49.029062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:49.029062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:49.029062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:49.029062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:49.029062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:49.029062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:49.029062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:49.029062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:49.029062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:49.029062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:49.029062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:49.029062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:49.031615 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 204/420] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:49.038588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c9cd7-a435-44c2-8f4f-a331ae4caf29 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:49.093104 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:49.166446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c9cd7-a435-44c2-8f4f-a331ae4caf29 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:49.203431 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead94f85-fb28-49f8-8341-165e08759faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:49.204058 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead94f85-fb28-49f8-8341-165e08759faf None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:49.204889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead94f85-fb28-49f8-8341-165e08759faf None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:49.205173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead94f85-fb28-49f8-8341-165e08759faf None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:49.205861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead94f85-fb28-49f8-8341-165e08759faf None admin] RBAC: Adding query filter params (name=member) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:49.206161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead94f85-fb28-49f8-8341-165e08759faf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:49.207604 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead94f85-fb28-49f8-8341-165e08759faf None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:49.210852 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7c76e-01ff-4d1d-ba33-c83a09af28f3 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:49.218161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 217/421] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:49.227876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead94f85-fb28-49f8-8341-165e08759faf None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:49.345137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed7c76e-01ff-4d1d-ba33-c83a09af28f3 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:49.377555 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:49.382541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead94f85-fb28-49f8-8341-165e08759faf None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:49.392753 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359c071-b056-460f-a802-3306981d839a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:49.393401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359c071-b056-460f-a802-3306981d839a None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:49.393678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359c071-b056-460f-a802-3306981d839a None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:49.393946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359c071-b056-460f-a802-3306981d839a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:49.397657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:49.411314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359c071-b056-460f-a802-3306981d839a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:49.418325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-3359c071-b056-460f-a802-3306981d839a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:49.418325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:49.418325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:49.418325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:49.418325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:49.418325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:49.418325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:49.418325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:49.418325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:49.418325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:49.418325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:49.418325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:49.418325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:49.418325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:49.418325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:49.418325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:49.419275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:49.419275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:49.419275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:49.419275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:49.419275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:49.419275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:49.419275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:49.419275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:49.419275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:49.419275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:49.419275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:49.419275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:49.419275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:49.419275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:49.419275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:49.419275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:49.420103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:49.420103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:49.420103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:49.420103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:49.420103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:49.420103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:49.420103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:49.420103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:49.420103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:49.420103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:49.420103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:49.420103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:49.420103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 205/422] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:49.432602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359c071-b056-460f-a802-3306981d839a None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:49.449229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c01ce7-85a2-4522-b1bc-e7be3deb9bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:49.450190 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c01ce7-85a2-4522-b1bc-e7be3deb9bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:49.450668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c01ce7-85a2-4522-b1bc-e7be3deb9bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:49.451276 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c01ce7-85a2-4522-b1bc-e7be3deb9bc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:49.572684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c01ce7-85a2-4522-b1bc-e7be3deb9bc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:49.574169 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c01ce7-85a2-4522-b1bc-e7be3deb9bc5 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:49.630050 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359c071-b056-460f-a802-3306981d839a None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:49.711575 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 218/423] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:49.721519 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c01ce7-85a2-4522-b1bc-e7be3deb9bc5 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:49.736830 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c2168-a214-4b30-a989-613911e4992e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:49.737295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:49.737473 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c2168-a214-4b30-a989-613911e4992e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:49.737717 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c2168-a214-4b30-a989-613911e4992e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:49.737984 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c2168-a214-4b30-a989-613911e4992e None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:49.738651 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c2168-a214-4b30-a989-613911e4992e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:49.739164 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c2168-a214-4b30-a989-613911e4992e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:49.740875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c2168-a214-4b30-a989-613911e4992e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:49.751346 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 206/424] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:49.756953 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c2168-a214-4b30-a989-613911e4992e None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:49.827414 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:49.920344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c01ce7-85a2-4522-b1bc-e7be3deb9bc5 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:49.950128 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c2168-a214-4b30-a989-613911e4992e None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:49.982547 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4fa8c-d78f-4cb1-89b8-4b257b8f17c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:49.983223 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4fa8c-d78f-4cb1-89b8-4b257b8f17c2 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:49.983502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4fa8c-d78f-4cb1-89b8-4b257b8f17c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:49.983775 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf4fa8c-d78f-4cb1-89b8-4b257b8f17c2 None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:49.986409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4fa8c-d78f-4cb1-89b8-4b257b8f17c2 None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:49.987962 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf4fa8c-d78f-4cb1-89b8-4b257b8f17c2 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:49.989706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 219/425] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:49 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:49.995347 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4fa8c-d78f-4cb1-89b8-4b257b8f17c2 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:50.018907 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334651eb-9886-4c5c-84c7-bb96176bafdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:50.019528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334651eb-9886-4c5c-84c7-bb96176bafdb None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.019805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334651eb-9886-4c5c-84c7-bb96176bafdb None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.020072 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334651eb-9886-4c5c-84c7-bb96176bafdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.136892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:50.161061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334651eb-9886-4c5c-84c7-bb96176bafdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:50.162534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334651eb-9886-4c5c-84c7-bb96176bafdb None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:50.185645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf4fa8c-d78f-4cb1-89b8-4b257b8f17c2 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:50.222199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:50.252619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45d50d-9548-4dd1-a3ef-9318ba25940f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:50.253262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45d50d-9548-4dd1-a3ef-9318ba25940f None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.253537 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45d50d-9548-4dd1-a3ef-9318ba25940f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.253808 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45d50d-9548-4dd1-a3ef-9318ba25940f None admin] PATH_INFO: `/v3/groups/f7bfc768836142d0b409afa8b8c12313` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.254980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45d50d-9548-4dd1-a3ef-9318ba25940f None admin] RBAC: Authorizing `identity:get_group(group_id=f7bfc768836142d0b409afa8b8c12313)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:50.256538 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45d50d-9548-4dd1-a3ef-9318ba25940f None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:50.258685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 220/426] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:49 2022] GET /identity/v3/groups/f7bfc768836142d0b409afa8b8c12313 => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:50.264905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45d50d-9548-4dd1-a3ef-9318ba25940f None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:50.346689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 207/427] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 14:44:50.354512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334651eb-9886-4c5c-84c7-bb96176bafdb None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:50.498072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45d50d-9548-4dd1-a3ef-9318ba25940f None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:50.530021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334651eb-9886-4c5c-84c7-bb96176bafdb None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:50.560724 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1db1a9c-4dec-4e90-9eda-f36a7ca7d137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:50.561342 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db1a9c-4dec-4e90-9eda-f36a7ca7d137 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.561622 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db1a9c-4dec-4e90-9eda-f36a7ca7d137 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.561894 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1db1a9c-4dec-4e90-9eda-f36a7ca7d137 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.562627 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db1a9c-4dec-4e90-9eda-f36a7ca7d137 None admin] RBAC: Adding query filter params (group.id=f7bfc768836142d0b409afa8b8c12313, role.id=7d3cf58c1ad2407c9f8c60ba99b2006d, scope.project.id=886ef67601844188bb0aa19dcf32dea7) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:50.562965 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db1a9c-4dec-4e90-9eda-f36a7ca7d137 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:50.564455 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1db1a9c-4dec-4e90-9eda-f36a7ca7d137 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:50.578988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 221/428] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:50 2022] GET /identity/v3/role_assignments?group.id=f7bfc768836142d0b409afa8b8c12313&scope.project.id=886ef67601844188bb0aa19dcf32dea7&role.id=7d3cf58c1ad2407c9f8c60ba99b2006d => generated 611 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:50.590455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba22e81-a8ee-4d3b-841d-3bc8b618a6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:50.591239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba22e81-a8ee-4d3b-841d-3bc8b618a6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.591870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba22e81-a8ee-4d3b-841d-3bc8b618a6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.592152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba22e81-a8ee-4d3b-841d-3bc8b618a6a3 None admin] PATH_INFO: `/v3/projects/886ef67601844188bb0aa19dcf32dea7` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.593079 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb22d8-b13a-4197-9b99-36a471250dac None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.593364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb22d8-b13a-4197-9b99-36a471250dac None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.593478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba22e81-a8ee-4d3b-841d-3bc8b618a6a3 None admin] RBAC: Authorizing `identity:get_project(project_id=886ef67601844188bb0aa19dcf32dea7)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:50.593639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb22d8-b13a-4197-9b99-36a471250dac None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.594861 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba22e81-a8ee-4d3b-841d-3bc8b618a6a3 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:50.597979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 208/430] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:50 2022] GET /identity/v3/projects/886ef67601844188bb0aa19dcf32dea7 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:50.598819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 222/430] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:44:50.601745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fb4b3-11fe-4e69-84d5-09136e32825c None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.602108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fb4b3-11fe-4e69-84d5-09136e32825c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.602446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fb4b3-11fe-4e69-84d5-09136e32825c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.604361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb22d8-b13a-4197-9b99-36a471250dac None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:50.625723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:50.632902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-f72fb4b3-11fe-4e69-84d5-09136e32825c None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:50.634138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f72fb4b3-11fe-4e69-84d5-09136e32825c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:50.645603 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:50.773273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 209/431] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:50.803048 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb22d8-b13a-4197-9b99-36a471250dac None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:50.843004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc297c1-c1a1-45f4-a3cc-321577060485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:50.844178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc297c1-c1a1-45f4-a3cc-321577060485 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.844655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc297c1-c1a1-45f4-a3cc-321577060485 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.845161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc297c1-c1a1-45f4-a3cc-321577060485 None admin] PATH_INFO: `/v3/groups/f7bfc768836142d0b409afa8b8c12313` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.846841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc297c1-c1a1-45f4-a3cc-321577060485 None admin] RBAC: Authorizing `identity:get_group(group_id=f7bfc768836142d0b409afa8b8c12313)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:50.848490 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc297c1-c1a1-45f4-a3cc-321577060485 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:50.850615 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 223/432] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:50 2022] GET /identity/v3/groups/f7bfc768836142d0b409afa8b8c12313 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:50.856599 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fb4b3-11fe-4e69-84d5-09136e32825c None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:50.956652 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1b7ee-7abe-4e86-a2a8-50778807715e None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.957278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1b7ee-7abe-4e86-a2a8-50778807715e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.957780 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1b7ee-7abe-4e86-a2a8-50778807715e None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.959419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 224/433] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:44:50.964946 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb1605-108b-4018-b2c9-e2c9fb7b02fe None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:50.965230 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb1605-108b-4018-b2c9-e2c9fb7b02fe None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:50.965504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb1605-108b-4018-b2c9-e2c9fb7b02fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:50.973214 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fb4b3-11fe-4e69-84d5-09136e32825c None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:50.986993 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-8adb1605-108b-4018-b2c9-e2c9fb7b02fe None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:50.988185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-8adb1605-108b-4018-b2c9-e2c9fb7b02fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:51.013597 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6d526-33a1-4be0-bb93-65df81a3691b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:51.014587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6d526-33a1-4be0-bb93-65df81a3691b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.015115 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6d526-33a1-4be0-bb93-65df81a3691b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.015603 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6d526-33a1-4be0-bb93-65df81a3691b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.016634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6d526-33a1-4be0-bb93-65df81a3691b None admin] RBAC: Adding query filter params (group.id=f7bfc768836142d0b409afa8b8c12313, role.id=1b632105fdba460b809a2ecd1d186409, scope.project.id=886ef67601844188bb0aa19dcf32dea7) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:51.017301 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6d526-33a1-4be0-bb93-65df81a3691b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:51.019080 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6d526-33a1-4be0-bb93-65df81a3691b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:51.026064 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 210/434] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:50 2022] GET /identity/v3/role_assignments?group.id=f7bfc768836142d0b409afa8b8c12313&scope.project.id=886ef67601844188bb0aa19dcf32dea7&role.id=1b632105fdba460b809a2ecd1d186409 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:51.080407 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 225/435] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:51.101127 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6d526-33a1-4be0-bb93-65df81a3691b None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:51.115349 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:51.236993 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6d526-33a1-4be0-bb93-65df81a3691b None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:51.275942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621ba9b-3fe5-4cbc-9d20-59584f6fae38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:51.276648 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621ba9b-3fe5-4cbc-9d20-59584f6fae38 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.276865 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621ba9b-3fe5-4cbc-9d20-59584f6fae38 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.277137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621ba9b-3fe5-4cbc-9d20-59584f6fae38 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.289638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621ba9b-3fe5-4cbc-9d20-59584f6fae38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:51.295330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-5621ba9b-3fe5-4cbc-9d20-59584f6fae38 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:51.295330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:51.295330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:51.295330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:51.295330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:51.295330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:51.295330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:51.295330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:51.295330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:51.295330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:51.295330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:51.295330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:51.295330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:51.295330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:51.295330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:51.295330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:51.296237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:51.296237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:51.296237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:51.296237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:51.296237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:51.296237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:51.296237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:51.296237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:51.296237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:51.296237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:51.296237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:51.296237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:51.296237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:51.296237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:51.296237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:51.296237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:51.297018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:51.297018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:51.297018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:51.297018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:51.297018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:51.297018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:51.297018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:51.297018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:51.297018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:51.297018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:51.297018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:51.297018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:51.297018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 211/436] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:51.303120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb1605-108b-4018-b2c9-e2c9fb7b02fe None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:51.316624 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:51.400093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621ba9b-3fe5-4cbc-9d20-59584f6fae38 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:51.433761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb1605-108b-4018-b2c9-e2c9fb7b02fe None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:51.485684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443d6eb-aef8-481b-ad8c-bc583de731ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:51.486592 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443d6eb-aef8-481b-ad8c-bc583de731ef None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.486951 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443d6eb-aef8-481b-ad8c-bc583de731ef None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.487228 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443d6eb-aef8-481b-ad8c-bc583de731ef None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.487909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443d6eb-aef8-481b-ad8c-bc583de731ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:51.488216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443d6eb-aef8-481b-ad8c-bc583de731ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:51.489711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443d6eb-aef8-481b-ad8c-bc583de731ef None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:51.503909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:51.504137 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 226/437] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:51.507428 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443d6eb-aef8-481b-ad8c-bc583de731ef None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:51.554185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621ba9b-3fe5-4cbc-9d20-59584f6fae38 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:51.600607 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e5511-ea78-4282-b79c-fd1a126cc3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:51.600946 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:51.601245 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e5511-ea78-4282-b79c-fd1a126cc3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.601519 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e5511-ea78-4282-b79c-fd1a126cc3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.601780 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e5511-ea78-4282-b79c-fd1a126cc3e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.614786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e5511-ea78-4282-b79c-fd1a126cc3e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:51.621796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-ac0e5511-ea78-4282-b79c-fd1a126cc3e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:51.621796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:51.621796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:51.621796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:51.621796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:51.621796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:51.621796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:51.621796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:51.621796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:51.621796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:51.621796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:51.621796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:51.621796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:51.621796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:51.621796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:51.621796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:51.622945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:51.622945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:51.622945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:51.622945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:51.622945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:51.622945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:51.622945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:51.622945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:51.622945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:51.622945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:51.622945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:51.622945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:51.622945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:51.622945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:51.622945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:51.622945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:51.623884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:51.623884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:51.623884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:51.623884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:51.623884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:51.623884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:51.623884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:51.623884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:51.623884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:51.623884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:51.623884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 14:44:51.623884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:51.625913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 212/438] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:51.632285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e5511-ea78-4282-b79c-fd1a126cc3e0 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:51.651472 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443d6eb-aef8-481b-ad8c-bc583de731ef None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:51.693992 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ebd86-feec-4374-8837-5d8a86780fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:51.694617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ebd86-feec-4374-8837-5d8a86780fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.694929 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ebd86-feec-4374-8837-5d8a86780fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.695204 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ebd86-feec-4374-8837-5d8a86780fb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.745282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e5511-ea78-4282-b79c-fd1a126cc3e0 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:51.771656 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:51.772729 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ebd86-feec-4374-8837-5d8a86780fb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:51.774398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ebd86-feec-4374-8837-5d8a86780fb6 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:51.780637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d3ecf-4883-436a-96ba-815be1269b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:51.781246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d3ecf-4883-436a-96ba-815be1269b62 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.781526 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d3ecf-4883-436a-96ba-815be1269b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.781848 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d3ecf-4883-436a-96ba-815be1269b62 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.782603 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d3ecf-4883-436a-96ba-815be1269b62 None admin] RBAC: Adding query filter params (name=member) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:51.783004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d3ecf-4883-436a-96ba-815be1269b62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:51.784377 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d3ecf-4883-436a-96ba-815be1269b62 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:51.792764 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 213/439] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:44:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:51.794090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390a015-f1a3-4da4-a64d-2567110be2e0 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.794375 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390a015-f1a3-4da4-a64d-2567110be2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.794647 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c390a015-f1a3-4da4-a64d-2567110be2e0 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.796138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 214/440] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:51.798288 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390a015-f1a3-4da4-a64d-2567110be2e0 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:51.842308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:51.885709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 227/441] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 14:44:51.887006 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f6fd3-ea4d-47e1-85e6-e84ecc1aaffa None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.887284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f6fd3-ea4d-47e1-85e6-e84ecc1aaffa None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.887551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f6fd3-ea4d-47e1-85e6-e84ecc1aaffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.904448 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c390a015-f1a3-4da4-a64d-2567110be2e0 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:51.905691 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-b20f6fd3-ea4d-47e1-85e6-e84ecc1aaffa None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:51.906912 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b20f6fd3-ea4d-47e1-85e6-e84ecc1aaffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:51.937528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b508cef-7830-406a-9092-b0025377a282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:51.938119 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b508cef-7830-406a-9092-b0025377a282 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:51.938394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b508cef-7830-406a-9092-b0025377a282 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:51.939201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b508cef-7830-406a-9092-b0025377a282 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:51.973136 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 228/442] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:51.974041 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f6fd3-ea4d-47e1-85e6-e84ecc1aaffa None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:51.999760 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:52.011936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b508cef-7830-406a-9092-b0025377a282 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:52.013457 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b508cef-7830-406a-9092-b0025377a282 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:52.088340 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:52.092465 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f6fd3-ea4d-47e1-85e6-e84ecc1aaffa None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:52.118305 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 215/443] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:44:52.123852 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b508cef-7830-406a-9092-b0025377a282 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:52.135943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b914ee0-9745-4b16-b763-8b4b3f6ae05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:52.136584 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b914ee0-9745-4b16-b763-8b4b3f6ae05b None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:52.136862 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b914ee0-9745-4b16-b763-8b4b3f6ae05b None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:52.137132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b914ee0-9745-4b16-b763-8b4b3f6ae05b None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:52.138418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b914ee0-9745-4b16-b763-8b4b3f6ae05b None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:52.140284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b914ee0-9745-4b16-b763-8b4b3f6ae05b None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:52.146350 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 229/444] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:51 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:52.147150 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b914ee0-9745-4b16-b763-8b4b3f6ae05b None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:52.226792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:52.238273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b508cef-7830-406a-9092-b0025377a282 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:52.253394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b914ee0-9745-4b16-b763-8b4b3f6ae05b None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:52.273344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b3cbd-7de9-444a-8b46-372d4e1e9d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:52.274170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b3cbd-7de9-444a-8b46-372d4e1e9d0d None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:52.274441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b3cbd-7de9-444a-8b46-372d4e1e9d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:52.274712 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b3cbd-7de9-444a-8b46-372d4e1e9d0d None admin] PATH_INFO: `/v3/projects/45e79ea1241346c58bf04c1494b3dc7d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:52.275837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b3cbd-7de9-444a-8b46-372d4e1e9d0d None admin] RBAC: Authorizing `identity:get_project(project_id=45e79ea1241346c58bf04c1494b3dc7d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:52.277270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b3cbd-7de9-444a-8b46-372d4e1e9d0d None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:52.279209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 216/445] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:52 2022] GET /identity/v3/projects/45e79ea1241346c58bf04c1494b3dc7d => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:52.279960 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b3cbd-7de9-444a-8b46-372d4e1e9d0d None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:52.291867 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764e5b6-8b72-457f-bfad-34b3b73ceaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:52.292581 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764e5b6-8b72-457f-bfad-34b3b73ceaa8 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:52.292863 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764e5b6-8b72-457f-bfad-34b3b73ceaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:52.293142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b764e5b6-8b72-457f-bfad-34b3b73ceaa8 None admin] PATH_INFO: `/v3/groups/f7bfc768836142d0b409afa8b8c12313` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:52.294278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764e5b6-8b72-457f-bfad-34b3b73ceaa8 None admin] RBAC: Authorizing `identity:get_group(group_id=f7bfc768836142d0b409afa8b8c12313)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:52.295737 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b764e5b6-8b72-457f-bfad-34b3b73ceaa8 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:52.297298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 230/446] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:52 2022] GET /identity/v3/groups/f7bfc768836142d0b409afa8b8c12313 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:52.299327 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764e5b6-8b72-457f-bfad-34b3b73ceaa8 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:52.337947 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:52.384481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b3cbd-7de9-444a-8b46-372d4e1e9d0d None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:52.403953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b764e5b6-8b72-457f-bfad-34b3b73ceaa8 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:52.423261 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395264d-e0a5-4ed4-9cc5-ab8c238e2a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:52.423889 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395264d-e0a5-4ed4-9cc5-ab8c238e2a4f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:52.424176 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395264d-e0a5-4ed4-9cc5-ab8c238e2a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:52.424439 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395264d-e0a5-4ed4-9cc5-ab8c238e2a4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:52.435092 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395264d-e0a5-4ed4-9cc5-ab8c238e2a4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:52.439810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70d6b4-f93b-4364-ab19-672ba51a3c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:52.440435 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70d6b4-f93b-4364-ab19-672ba51a3c08 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:52.440534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-c395264d-e0a5-4ed4-9cc5-ab8c238e2a4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:52.440534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:52.440534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:52.440534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:52.440534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:52.440534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:52.440534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:52.440534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:52.440534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:52.440534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:52.440534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:52.440534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:52.440534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:52.440534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:52.440534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:52.440534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:52.441517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:52.441517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:52.441517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:52.441517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:52.441517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:52.441517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:52.441517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:52.441517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:52.441517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:52.441517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:52.441517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:52.441517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:52.441517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:52.441517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:52.441517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:52.441517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:52.442335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:52.442335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:52.442335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:52.442335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:52.442335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:52.442335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:52.442335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:52.442335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:52.442335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:52.442335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:52.442335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:44:52.442335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:52.442976 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70d6b4-f93b-4364-ab19-672ba51a3c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:52.442976 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70d6b4-f93b-4364-ab19-672ba51a3c08 None admin] PATH_INFO: `/v3/groups/f7bfc768836142d0b409afa8b8c12313` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:52.443093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 217/447] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:44:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:52.443093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395264d-e0a5-4ed4-9cc5-ab8c238e2a4f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:52.443609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa70d6b4-f93b-4364-ab19-672ba51a3c08 None admin] RBAC: Authorizing `identity:get_group(group_id=f7bfc768836142d0b409afa8b8c12313)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:52.445839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa70d6b4-f93b-4364-ab19-672ba51a3c08 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:52.446546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 231/448] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:52 2022] GET /identity/v3/groups/f7bfc768836142d0b409afa8b8c12313 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:52.452048 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70d6b4-f93b-4364-ab19-672ba51a3c08 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:52.463654 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:52.544192 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395264d-e0a5-4ed4-9cc5-ab8c238e2a4f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:52.553358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70d6b4-f93b-4364-ab19-672ba51a3c08 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:52.554906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:52.578101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a7013-01e7-429b-8a6e-c836f520fcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:52.578704 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a7013-01e7-429b-8a6e-c836f520fcb7 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:52.579010 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a7013-01e7-429b-8a6e-c836f520fcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:52.579317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a7013-01e7-429b-8a6e-c836f520fcb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:52.580553 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a7013-01e7-429b-8a6e-c836f520fcb7 None admin] RBAC: Adding query filter params (group.id=f7bfc768836142d0b409afa8b8c12313, role.id=1b632105fdba460b809a2ecd1d186409, scope.project.id=45e79ea1241346c58bf04c1494b3dc7d) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:52.580862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a7013-01e7-429b-8a6e-c836f520fcb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:52.582319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a7013-01e7-429b-8a6e-c836f520fcb7 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:52.588008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 218/449] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:52 2022] GET /identity/v3/role_assignments?group.id=f7bfc768836142d0b409afa8b8c12313&scope.project.id=45e79ea1241346c58bf04c1494b3dc7d&role.id=1b632105fdba460b809a2ecd1d186409 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:52.588803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a7013-01e7-429b-8a6e-c836f520fcb7 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:52.597619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cae351-2ddf-48d8-83a6-b548a39d5290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:52.598992 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cae351-2ddf-48d8-83a6-b548a39d5290 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:52.599347 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cae351-2ddf-48d8-83a6-b548a39d5290 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:52.599698 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cae351-2ddf-48d8-83a6-b548a39d5290 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:52.600471 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cae351-2ddf-48d8-83a6-b548a39d5290 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:52.601320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cae351-2ddf-48d8-83a6-b548a39d5290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:52.603388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cae351-2ddf-48d8-83a6-b548a39d5290 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:52.614871 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 232/450] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:44:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:52.620649 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cae351-2ddf-48d8-83a6-b548a39d5290 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:52.683104 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:52.706611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a7013-01e7-429b-8a6e-c836f520fcb7 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:52.738657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cae351-2ddf-48d8-83a6-b548a39d5290 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:52.745860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83c8a9-0342-4db5-929c-e02591f6dea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:52.746475 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83c8a9-0342-4db5-929c-e02591f6dea9 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:52.746780 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83c8a9-0342-4db5-929c-e02591f6dea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:52.747048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83c8a9-0342-4db5-929c-e02591f6dea9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:52.747779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83c8a9-0342-4db5-929c-e02591f6dea9 None admin] RBAC: Adding query filter params (group.id=f7bfc768836142d0b409afa8b8c12313, role.id=7d3cf58c1ad2407c9f8c60ba99b2006d, scope.project.id=45e79ea1241346c58bf04c1494b3dc7d) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:52.748083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83c8a9-0342-4db5-929c-e02591f6dea9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:52.749557 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83c8a9-0342-4db5-929c-e02591f6dea9 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:52.758765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 219/451] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:52 2022] GET /identity/v3/role_assignments?group.id=f7bfc768836142d0b409afa8b8c12313&scope.project.id=45e79ea1241346c58bf04c1494b3dc7d&role.id=7d3cf58c1ad2407c9f8c60ba99b2006d => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:52.788696 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714bf35-d120-4b4d-8874-bb106f0ec16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:52.789608 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714bf35-d120-4b4d-8874-bb106f0ec16c None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:52.789887 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714bf35-d120-4b4d-8874-bb106f0ec16c None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:52.790157 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714bf35-d120-4b4d-8874-bb106f0ec16c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:52.831400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:52.877620 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714bf35-d120-4b4d-8874-bb106f0ec16c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:52.879153 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714bf35-d120-4b4d-8874-bb106f0ec16c None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:52.985381 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 233/452] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:44:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 14:44:52.993145 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83c8a9-0342-4db5-929c-e02591f6dea9 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:53.101238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83c8a9-0342-4db5-929c-e02591f6dea9 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:53.136807 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2d800-c21e-40a1-9b65-450531614a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:53.137441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2d800-c21e-40a1-9b65-450531614a03 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:53.137723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2d800-c21e-40a1-9b65-450531614a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:53.137997 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2d800-c21e-40a1-9b65-450531614a03 None admin] PATH_INFO: `/v3/projects/82e4fd5adbe54235a872491a4adc6897` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:53.139151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2d800-c21e-40a1-9b65-450531614a03 None admin] RBAC: Authorizing `identity:get_project(project_id=82e4fd5adbe54235a872491a4adc6897)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:53.140616 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2d800-c21e-40a1-9b65-450531614a03 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:53.142278 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 220/453] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:44:52 2022] GET /identity/v3/projects/82e4fd5adbe54235a872491a4adc6897 => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:53.148114 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714bf35-d120-4b4d-8874-bb106f0ec16c None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:53.210669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:53.251426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714bf35-d120-4b4d-8874-bb106f0ec16c None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:53.285513 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4d60a-7308-4fbf-a62f-5bc96d6aca89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:53.286146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4d60a-7308-4fbf-a62f-5bc96d6aca89 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:53.286424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4d60a-7308-4fbf-a62f-5bc96d6aca89 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:53.286692 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4d60a-7308-4fbf-a62f-5bc96d6aca89 None admin] PATH_INFO: `/v3/groups/abad01993c6c45c3b199eb371488b694` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:53.287815 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4d60a-7308-4fbf-a62f-5bc96d6aca89 None admin] RBAC: Authorizing `identity:get_group(group_id=abad01993c6c45c3b199eb371488b694)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:53.289250 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4d60a-7308-4fbf-a62f-5bc96d6aca89 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:53.290862 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 234/454] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Sep 22 14:44:53 2022] GET /identity/v3/groups/abad01993c6c45c3b199eb371488b694 => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:53.296750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2d800-c21e-40a1-9b65-450531614a03 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:53.313317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:53.397597 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2d800-c21e-40a1-9b65-450531614a03 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:53.430579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f4f0a-444f-42a0-8f57-0dd4c8828b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:53.431196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f4f0a-444f-42a0-8f57-0dd4c8828b00 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:53.431474 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f4f0a-444f-42a0-8f57-0dd4c8828b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:53.431743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f4f0a-444f-42a0-8f57-0dd4c8828b00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:53.432459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f4f0a-444f-42a0-8f57-0dd4c8828b00 None admin] RBAC: Adding query filter params (group.id=abad01993c6c45c3b199eb371488b694, role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=82e4fd5adbe54235a872491a4adc6897) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:53.432760 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f4f0a-444f-42a0-8f57-0dd4c8828b00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:53.434178 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f4f0a-444f-42a0-8f57-0dd4c8828b00 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:53.439767 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 221/455] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Sep 22 14:44:53 2022] GET /identity/v3/role_assignments?group.id=abad01993c6c45c3b199eb371488b694&scope.project.id=82e4fd5adbe54235a872491a4adc6897&role.id=32c8a81e339a462e973e76bb528ef35a => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:56.179363 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765bd45-ca4c-4b1a-84d3-b1090a19353d None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.179874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765bd45-ca4c-4b1a-84d3-b1090a19353d None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.179948 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765bd45-ca4c-4b1a-84d3-b1090a19353d None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.181719 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 235/456] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:56.186694 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0350f3-7549-4de0-aa77-15bdd6707826 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.187097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0350f3-7549-4de0-aa77-15bdd6707826 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.187257 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0350f3-7549-4de0-aa77-15bdd6707826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.215695 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-5d0350f3-7549-4de0-aa77-15bdd6707826 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:56.216870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-5d0350f3-7549-4de0-aa77-15bdd6707826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:56.314225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 222/457] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:56.444664 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990fe8a-ea17-4129-a07f-6c19bf1e5957 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.444862 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990fe8a-ea17-4129-a07f-6c19bf1e5957 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.445378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990fe8a-ea17-4129-a07f-6c19bf1e5957 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.448119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 236/458] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:56.456789 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe074f7-8ecb-4f0a-a77f-84b888c63d6d None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.457546 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe074f7-8ecb-4f0a-a77f-84b888c63d6d None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.458213 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe074f7-8ecb-4f0a-a77f-84b888c63d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.494281 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-fbe074f7-8ecb-4f0a-a77f-84b888c63d6d None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:56.498698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-fbe074f7-8ecb-4f0a-a77f-84b888c63d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:56.501216 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:56.538558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914006a-a186-4ffa-9234-2a33ac2c7388 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.539227 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914006a-a186-4ffa-9234-2a33ac2c7388 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.539701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6914006a-a186-4ffa-9234-2a33ac2c7388 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.542583 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 237/459] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:56.546962 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99361ac3-5701-4f64-8040-a6c06ec384cd None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.547519 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99361ac3-5701-4f64-8040-a6c06ec384cd None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.548070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99361ac3-5701-4f64-8040-a6c06ec384cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.566651 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-99361ac3-5701-4f64-8040-a6c06ec384cd None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:56.568345 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-99361ac3-5701-4f64-8040-a6c06ec384cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:56.614158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 223/460] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:56.658395 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 238/461] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:44:56.671317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe074f7-8ecb-4f0a-a77f-84b888c63d6d None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:56.777683 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe074f7-8ecb-4f0a-a77f-84b888c63d6d None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:56.819407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfe3da-2b64-4e19-96d2-e5fa20a6ac22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:56.820465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfe3da-2b64-4e19-96d2-e5fa20a6ac22 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:56.820942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfe3da-2b64-4e19-96d2-e5fa20a6ac22 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:56.821628 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfe3da-2b64-4e19-96d2-e5fa20a6ac22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:56.828725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:56.833929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbfe3da-2b64-4e19-96d2-e5fa20a6ac22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:56.835689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbfe3da-2b64-4e19-96d2-e5fa20a6ac22 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:56.854913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-4cbfe3da-2b64-4e19-96d2-e5fa20a6ac22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:44:56.854913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:56.854913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:56.854913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:56.854913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:56.854913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:56.854913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:56.854913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:56.854913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:56.854913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:56.854913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:56.854913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:56.854913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:44:56.854913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:44:56.854913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:44:56.854913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:44:56.855888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:56.855888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:56.855888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:56.855888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:56.855888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:56.855888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:56.855888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:56.855888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:56.855888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:56.855888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:56.855888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:56.855888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:56.855888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:56.855888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:56.855888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:56.855888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:56.855888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:44:56.856745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:44:56.856745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:44:56.856745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:56.858826 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 224/462] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:56.867812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99361ac3-5701-4f64-8040-a6c06ec384cd None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:56.947932 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:56.973403 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfe3da-2b64-4e19-96d2-e5fa20a6ac22 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:57.007782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99361ac3-5701-4f64-8040-a6c06ec384cd None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:57.075211 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47b3d4-cfc4-4dde-910a-959b75966383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:57.076353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47b3d4-cfc4-4dde-910a-959b75966383 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.077584 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47b3d4-cfc4-4dde-910a-959b75966383 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.078666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47b3d4-cfc4-4dde-910a-959b75966383 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.080175 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47b3d4-cfc4-4dde-910a-959b75966383 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:57.082522 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47b3d4-cfc4-4dde-910a-959b75966383 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:57.084082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47b3d4-cfc4-4dde-910a-959b75966383 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:57.128509 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 239/463] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:57.130116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a106d3-1dca-4598-b72f-0b4ce867487f None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.130534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a106d3-1dca-4598-b72f-0b4ce867487f None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.131028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a106d3-1dca-4598-b72f-0b4ce867487f None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.132750 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 240/464] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:57.134121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a106d3-1dca-4598-b72f-0b4ce867487f None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:57.152350 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfe3da-2b64-4e19-96d2-e5fa20a6ac22 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:57.189882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e97aa4-7e66-458d-91cc-7bdb290a107a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:57.191246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e97aa4-7e66-458d-91cc-7bdb290a107a None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.191526 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e97aa4-7e66-458d-91cc-7bdb290a107a None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.191799 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e97aa4-7e66-458d-91cc-7bdb290a107a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.202450 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e97aa4-7e66-458d-91cc-7bdb290a107a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:57.203930 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e97aa4-7e66-458d-91cc-7bdb290a107a None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:57.218851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-a4e97aa4-7e66-458d-91cc-7bdb290a107a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:44:57.218851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:57.218851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:57.218851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:57.218851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:57.218851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:57.218851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:57.218851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:57.218851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:57.218851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:57.218851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:57.218851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:57.218851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:44:57.218851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:44:57.218851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:44:57.218851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:44:57.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:57.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:57.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:57.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:57.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:57.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:57.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:57.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:57.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:57.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:57.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:57.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:57.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:57.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:57.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:57.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:57.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:44:57.223860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:44:57.223860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:44:57.223860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:57.223860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 225/465] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:44:57.223860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279b7a9-dfc7-40f6-aa75-7d54fa81ce85 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.223860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279b7a9-dfc7-40f6-aa75-7d54fa81ce85 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.223860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279b7a9-dfc7-40f6-aa75-7d54fa81ce85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.240182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-d279b7a9-dfc7-40f6-aa75-7d54fa81ce85 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:57.241391 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d279b7a9-dfc7-40f6-aa75-7d54fa81ce85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:57.254310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a106d3-1dca-4598-b72f-0b4ce867487f None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:57.282944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:57.284201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:57.299139 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99b1b8-75f1-4596-a93e-351de439fc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:57.299754 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99b1b8-75f1-4596-a93e-351de439fc84 None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.300036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99b1b8-75f1-4596-a93e-351de439fc84 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.300309 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99b1b8-75f1-4596-a93e-351de439fc84 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.300973 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99b1b8-75f1-4596-a93e-351de439fc84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:57.302409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99b1b8-75f1-4596-a93e-351de439fc84 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:57.325070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 226/466] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:57.326307 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279b7a9-dfc7-40f6-aa75-7d54fa81ce85 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:57.339293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 241/467] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:57 2022] POST /identity/v3/users => generated 246 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:44:57.347460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99b1b8-75f1-4596-a93e-351de439fc84 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:57.541357 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279b7a9-dfc7-40f6-aa75-7d54fa81ce85 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:57.563881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99b1b8-75f1-4596-a93e-351de439fc84 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:57.581337 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708887b2-b2d2-4a4a-ac55-85d33e0a7b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:57.581951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708887b2-b2d2-4a4a-ac55-85d33e0a7b7b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.582232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708887b2-b2d2-4a4a-ac55-85d33e0a7b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.582499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708887b2-b2d2-4a4a-ac55-85d33e0a7b7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.594693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708887b2-b2d2-4a4a-ac55-85d33e0a7b7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:57.597256 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708887b2-b2d2-4a4a-ac55-85d33e0a7b7b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:57.607004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992be08b-5e10-4da3-84f8-444fe66b7ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:57.607843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992be08b-5e10-4da3-84f8-444fe66b7ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.608213 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992be08b-5e10-4da3-84f8-444fe66b7ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.608575 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992be08b-5e10-4da3-84f8-444fe66b7ae1 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.609325 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992be08b-5e10-4da3-84f8-444fe66b7ae1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:57.609694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992be08b-5e10-4da3-84f8-444fe66b7ae1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:57.611766 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992be08b-5e10-4da3-84f8-444fe66b7ae1 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:57.615460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-708887b2-b2d2-4a4a-ac55-85d33e0a7b7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:44:57.615460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:57.615460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:57.615460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:57.615460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:57.615460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:57.615460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:57.615460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:57.615460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:57.615460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:57.615460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:57.615460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:57.615460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:44:57.615460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:44:57.615460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:44:57.615460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:44:57.616389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:57.616389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:57.616389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:57.616389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:57.616389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:57.616389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:57.616389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:57.616389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:57.616389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:57.616389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:57.616389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:57.616389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:57.616389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:57.616389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:57.616389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:57.616389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:57.616389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:44:57.617236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:44:57.617236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:44:57.617236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:57.619432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 227/468] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:57.625485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708887b2-b2d2-4a4a-ac55-85d33e0a7b7b None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:57.627128 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 242/469] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:57.642661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992be08b-5e10-4da3-84f8-444fe66b7ae1 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:57.663674 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:57.695946 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:57.751511 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708887b2-b2d2-4a4a-ac55-85d33e0a7b7b None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:57.761599 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992be08b-5e10-4da3-84f8-444fe66b7ae1 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:57.788880 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8d733-c0ac-4fbe-8069-b4b20c20fe3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:57.789489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8d733-c0ac-4fbe-8069-b4b20c20fe3e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.789882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8d733-c0ac-4fbe-8069-b4b20c20fe3e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.790154 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8d733-c0ac-4fbe-8069-b4b20c20fe3e None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.790935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8d733-c0ac-4fbe-8069-b4b20c20fe3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:57.791237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8d733-c0ac-4fbe-8069-b4b20c20fe3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:57.792710 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8d733-c0ac-4fbe-8069-b4b20c20fe3e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:57.798966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf132f-812f-48ed-a3ce-0cf218f1900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:57.799614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf132f-812f-48ed-a3ce-0cf218f1900b None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.799886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf132f-812f-48ed-a3ce-0cf218f1900b None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.800151 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf132f-812f-48ed-a3ce-0cf218f1900b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.806169 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 228/470] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:44:57.807118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8d733-c0ac-4fbe-8069-b4b20c20fe3e None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:57.816903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf132f-812f-48ed-a3ce-0cf218f1900b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:57.818368 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf132f-812f-48ed-a3ce-0cf218f1900b None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:57.835319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-42cf132f-812f-48ed-a3ce-0cf218f1900b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:44:57.835319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:57.835319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:57.835319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:57.835319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:57.835319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:57.835319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:57.835319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:57.835319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:57.835319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:57.835319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:57.835319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:57.835319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:44:57.835319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:44:57.835319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:44:57.835319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:44:57.836237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:57.836237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:57.836237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:57.836237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:57.836237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:57.836237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:57.836237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:57.836237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:57.836237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:57.836237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:57.836237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:57.836237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:57.836237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:57.836237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:57.836237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:57.836237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:57.836237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:44:57.837075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:44:57.837075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:44:57.837075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:57.838590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 243/471] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:57.844647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68b946-d3e0-4904-a68d-b525ca4e2490 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.845505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68b946-d3e0-4904-a68d-b525ca4e2490 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.845766 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68b946-d3e0-4904-a68d-b525ca4e2490 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.847486 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 244/472] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:57.848498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68b946-d3e0-4904-a68d-b525ca4e2490 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:57.900233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:57.918029 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8d733-c0ac-4fbe-8069-b4b20c20fe3e None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:57.954250 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:57.959597 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c3460-470a-4d63-876b-40b3da03173a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:57.960211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c3460-470a-4d63-876b-40b3da03173a None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:57.960485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c3460-470a-4d63-876b-40b3da03173a None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:57.960756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c3460-470a-4d63-876b-40b3da03173a None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:57.961396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c3460-470a-4d63-876b-40b3da03173a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:57.962861 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c3460-470a-4d63-876b-40b3da03173a None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:57.966492 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68b946-d3e0-4904-a68d-b525ca4e2490 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:57.988890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 229/473] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:57 2022] POST /identity/v3/users => generated 248 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:57.989743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c3460-470a-4d63-876b-40b3da03173a None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:58.022960 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00070f-9cd8-49be-8f0b-f5ac3df4ca16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:58.024050 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00070f-9cd8-49be-8f0b-f5ac3df4ca16 None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:58.024437 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00070f-9cd8-49be-8f0b-f5ac3df4ca16 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:58.025316 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00070f-9cd8-49be-8f0b-f5ac3df4ca16 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:58.026471 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00070f-9cd8-49be-8f0b-f5ac3df4ca16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:58.029243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00070f-9cd8-49be-8f0b-f5ac3df4ca16 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:58.061322 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 245/474] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:57 2022] POST /identity/v3/users => generated 248 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:44:58.062905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ad62e-b62c-421b-a234-27c2f496e77f None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:58.063272 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ad62e-b62c-421b-a234-27c2f496e77f None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:58.064161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ad62e-b62c-421b-a234-27c2f496e77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:58.102969 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-205ad62e-b62c-421b-a234-27c2f496e77f None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:58.104180 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-205ad62e-b62c-421b-a234-27c2f496e77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:58.186948 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c3460-470a-4d63-876b-40b3da03173a None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:58.226320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 246/475] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:44:58.228157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62abc05-5e46-410e-90c1-38707703a241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:58.228790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62abc05-5e46-410e-90c1-38707703a241 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:58.229071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62abc05-5e46-410e-90c1-38707703a241 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:58.229341 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62abc05-5e46-410e-90c1-38707703a241 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:58.230050 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62abc05-5e46-410e-90c1-38707703a241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:58.230361 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62abc05-5e46-410e-90c1-38707703a241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:58.231931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62abc05-5e46-410e-90c1-38707703a241 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:58.242973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:58.247058 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 230/476] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:58.253541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ad62e-b62c-421b-a234-27c2f496e77f None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:58.357620 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:58.384024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ad62e-b62c-421b-a234-27c2f496e77f None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:58.423394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ee2fa-7d9f-4262-a639-d4b7340baee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:58.424309 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ee2fa-7d9f-4262-a639-d4b7340baee3 None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:58.424792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ee2fa-7d9f-4262-a639-d4b7340baee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:58.425240 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ee2fa-7d9f-4262-a639-d4b7340baee3 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:58.426068 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ee2fa-7d9f-4262-a639-d4b7340baee3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:58.427876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ee2fa-7d9f-4262-a639-d4b7340baee3 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:58.453284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 247/477] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:44:58 2022] POST /identity/v3/users => generated 249 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:44:58.514315 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62abc05-5e46-410e-90c1-38707703a241 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:58.660408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62abc05-5e46-410e-90c1-38707703a241 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:58.703100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729721c4-1a98-4494-8b64-97d2c8345f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:58.704074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729721c4-1a98-4494-8b64-97d2c8345f64 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:58.705095 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729721c4-1a98-4494-8b64-97d2c8345f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:58.705669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729721c4-1a98-4494-8b64-97d2c8345f64 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:58.706560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729721c4-1a98-4494-8b64-97d2c8345f64 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:58.708248 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729721c4-1a98-4494-8b64-97d2c8345f64 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:58.721288 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.sql.core [None req-729721c4-1a98-4494-8b64-97d2c8345f64 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 14:44:58.721288 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 22 14:44:58.721288 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [parameters: {'id': '8f62c01dfee648faa25a3438a254d9f4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 22 14:44:58.721288 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81454) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 14:44:58.739964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-729721c4-1a98-4494-8b64-97d2c8345f64 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 22 14:44:58.739964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:58.739964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 14:44:58.739964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 14:44:58.739964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 14:44:58.739964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 14:44:58.739964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 14:44:58.739964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 14:44:58.739964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 14:44:58.739964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application conn.query(q) Sep 22 14:44:58.739964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 14:44:58.739964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 14:44:58.739964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 14:44:58.739964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application result.read() Sep 22 14:44:58.739964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return ref.to_dict() Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 14:44:58.740898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application next(self.gen) Sep 22 14:44:58.741899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 14:44:58.741899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application yield resource Sep 22 14:44:58.741899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 14:44:58.741899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application next(self.gen) Sep 22 14:44:58.741899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 14:44:58.741899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 14:44:58.741899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 14:44:58.741899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 14:44:58.741899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 14:44:58.741899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise self.value Sep 22 14:44:58.741899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 14:44:58.741899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 14:44:58.741899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 14:44:58.741899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application session.commit() Sep 22 14:44:58.741899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 14:44:58.741899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 14:44:58.741899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 14:44:58.742777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 14:44:58.742777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 14:44:58.742777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self.session.flush() Sep 22 14:44:58.742777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 14:44:58.742777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 14:44:58.742777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 14:44:58.742777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 14:44:58.742777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 14:44:58.742777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application compat.raise_( Sep 22 14:44:58.742777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 14:44:58.742777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception Sep 22 14:44:58.742777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 14:44:58.742777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 14:44:58.742777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 14:44:58.742777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 14:44:58.742777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 14:44:58.742777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 14:44:58.743665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 14:44:58.743665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 14:44:58.743665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 22 14:44:58.743665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 22 14:44:58.743665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 14:44:58.743665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 14:44:58.743665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 14:44:58.743665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 14:44:58.743665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 14:44:58.743665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 14:44:58.743665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 14:44:58.743665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 14:44:58.743665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 14:44:58.743665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 14:44:58.743665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 14:44:58.743665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception Sep 22 14:44:58.744489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 14:44:58.744489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 14:44:58.744489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 14:44:58.744489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 14:44:58.744489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 14:44:58.744489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 14:44:58.744489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 14:44:58.744489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application conn.query(q) Sep 22 14:44:58.744489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 14:44:58.744489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 14:44:58.744489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 14:44:58.744489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application result.read() Sep 22 14:44:58.744489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 14:44:58.744489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 14:44:58.744489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 14:44:58.744489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 14:44:58.745260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 14:44:58.745260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 14:44:58.745260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 14:44:58.745260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 14:44:58.745260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 14:44:58.745260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 22 14:44:58.745260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [parameters: {'id': '8f62c01dfee648faa25a3438a254d9f4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 22 14:44:58.745260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 14:44:58.745260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:58.745260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:44:58.745260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:58.745260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:58.745260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:58.745260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:58.745260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:58.745260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:58.747355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:58.747355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:58.747355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:58.747355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:58.747355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:58.747355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:58.747355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 22 14:44:58.747355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 22 14:44:58.747355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:58.747355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:58.747355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 22 14:44:58.747355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 22 14:44:58.747355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 14:44:58.747355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 14:44:58.747355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 22 14:44:58.747355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:44:58.749528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 231/478] 10.222.0.26 () {60 vars in 1131 bytes} [Thu Sep 22 14:44:58 2022] POST /identity/v3/roles => generated 144 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:58.758920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ee2fa-7d9f-4262-a639-d4b7340baee3 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:58.871995 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ee2fa-7d9f-4262-a639-d4b7340baee3 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:58.903143 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:58.913419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ed65f-cdf7-43b4-85ed-609410c50e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:58.914001 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ed65f-cdf7-43b4-85ed-609410c50e21 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:58.914286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ed65f-cdf7-43b4-85ed-609410c50e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:58.914565 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ed65f-cdf7-43b4-85ed-609410c50e21 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:58.928580 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ed65f-cdf7-43b4-85ed-609410c50e21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:44:58.933957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-f57ed65f-cdf7-43b4-85ed-609410c50e21 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:58.933957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:44:58.933957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:44:58.933957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:44:58.933957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:44:58.933957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:44:58.933957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:44:58.933957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:44:58.933957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:44:58.933957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:44:58.933957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:44:58.933957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:44:58.933957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:44:58.933957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:44:58.933957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:44:58.933957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:44:58.934901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:44:58.934901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:44:58.934901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:44:58.934901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:44:58.934901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:44:58.934901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:44:58.934901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:44:58.934901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:44:58.934901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:44:58.934901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:44:58.934901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:44:58.934901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:44:58.934901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:44:58.934901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:44:58.934901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:44:58.934901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:44:58.935706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:44:58.935706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:44:58.935706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:44:58.935706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:44:58.935706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:44:58.935706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:44:58.935706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:44:58.935706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:44:58.935706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:44:58.935706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:44:58.935706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 14:44:58.935706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:44:58.935706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 248/479] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:44:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:44:58.943574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729721c4-1a98-4494-8b64-97d2c8345f64 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:44:59.026198 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:44:59.075238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729721c4-1a98-4494-8b64-97d2c8345f64 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:44:59.122714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c75eab-22a0-4451-9ce0-8f0572832a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:44:59.123360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c75eab-22a0-4451-9ce0-8f0572832a52 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:59.123636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c75eab-22a0-4451-9ce0-8f0572832a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:59.124017 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c75eab-22a0-4451-9ce0-8f0572832a52 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:59.124688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c75eab-22a0-4451-9ce0-8f0572832a52 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:44:59.124987 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c75eab-22a0-4451-9ce0-8f0572832a52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:44:59.126413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c75eab-22a0-4451-9ce0-8f0572832a52 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:44:59.138175 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 232/480] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:44:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:44:59.817735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1505039-5b4f-4c33-9bb4-b6134a034892 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:59.818684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1505039-5b4f-4c33-9bb4-b6134a034892 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:59.819223 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1505039-5b4f-4c33-9bb4-b6134a034892 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:59.821030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 249/481] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:44:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:44:59.826746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32924e11-fa27-4f02-a619-3d2ba492c75c None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:44:59.827567 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32924e11-fa27-4f02-a619-3d2ba492c75c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:44:59.828054 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32924e11-fa27-4f02-a619-3d2ba492c75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:44:59.851736 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-32924e11-fa27-4f02-a619-3d2ba492c75c None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:44:59.853269 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-32924e11-fa27-4f02-a619-3d2ba492c75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:44:59.945969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 233/482] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:44:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:00.366241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1505039-5b4f-4c33-9bb4-b6134a034892 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:00.507165 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1505039-5b4f-4c33-9bb4-b6134a034892 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:00.559359 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b846a4cb-33e9-4bf7-8346-f334eb6360f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:00.560008 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846a4cb-33e9-4bf7-8346-f334eb6360f3 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:00.560285 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846a4cb-33e9-4bf7-8346-f334eb6360f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:00.560558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b846a4cb-33e9-4bf7-8346-f334eb6360f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:00.574991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b846a4cb-33e9-4bf7-8346-f334eb6360f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:00.580504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-b846a4cb-33e9-4bf7-8346-f334eb6360f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:00.580504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:00.580504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:00.580504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:00.580504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:00.580504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:00.580504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:00.580504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:00.580504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:00.580504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:00.580504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:00.580504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:00.580504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:00.580504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:00.580504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:00.580504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:00.581390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:00.581390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:00.581390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:00.581390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:00.581390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:00.581390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:00.581390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:00.581390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:00.581390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:00.581390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:00.581390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:00.581390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:00.581390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:00.581390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:00.581390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:00.581390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:00.582271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:00.582271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:00.582271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:00.582271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:00.582271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:00.582271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:00.582271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:00.582271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:00.582271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:00.582271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:00.582271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:00.582271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:00.582271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 250/483] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:00.587603 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32924e11-fa27-4f02-a619-3d2ba492c75c None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:00.664807 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:00.725979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32924e11-fa27-4f02-a619-3d2ba492c75c None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:00.764867 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e094b-cfae-4137-a173-91443f06389d None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:00.765187 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e094b-cfae-4137-a173-91443f06389d None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:00.765400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e094b-cfae-4137-a173-91443f06389d None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:00.767835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 251/484] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:00.772336 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9fee5-0230-42ce-8be6-f6376acef2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:00.772936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdb987-d41a-47cc-8f2c-e8a7ceea4cfd None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:00.773218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdb987-d41a-47cc-8f2c-e8a7ceea4cfd None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:00.773496 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdb987-d41a-47cc-8f2c-e8a7ceea4cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:00.773869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9fee5-0230-42ce-8be6-f6376acef2ca None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:00.774340 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9fee5-0230-42ce-8be6-f6376acef2ca None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:00.774836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9fee5-0230-42ce-8be6-f6376acef2ca None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:00.775697 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9fee5-0230-42ce-8be6-f6376acef2ca None admin] RBAC: Adding query filter params (name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:00.776216 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9fee5-0230-42ce-8be6-f6376acef2ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:00.777774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9fee5-0230-42ce-8be6-f6376acef2ca None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:00.797085 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-78cdb987-d41a-47cc-8f2c-e8a7ceea4cfd None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:00.798347 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-78cdb987-d41a-47cc-8f2c-e8a7ceea4cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:00.800819 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 234/485] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:00.806504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9fee5-0230-42ce-8be6-f6376acef2ca None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:00.879289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:00.905832 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 252/486] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:00.995678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9fee5-0230-42ce-8be6-f6376acef2ca None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:01.002969 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ec330-ea6a-4d3f-82f8-3b5cbdd865c5 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.003257 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ec330-ea6a-4d3f-82f8-3b5cbdd865c5 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.003535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ec330-ea6a-4d3f-82f8-3b5cbdd865c5 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.005149 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 253/487] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:01.011751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaa0e2-e288-4df2-be2f-488a23233b19 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.012104 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaa0e2-e288-4df2-be2f-488a23233b19 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.012390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdaa0e2-e288-4df2-be2f-488a23233b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.024150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:01.039898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-2cdaa0e2-e288-4df2-be2f-488a23233b19 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:01.041753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-2cdaa0e2-e288-4df2-be2f-488a23233b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:01.055644 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f003d-aa22-4ff5-a619-c9b25b99b0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:01.056287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f003d-aa22-4ff5-a619-c9b25b99b0db None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.056564 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f003d-aa22-4ff5-a619-c9b25b99b0db None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.056835 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f003d-aa22-4ff5-a619-c9b25b99b0db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.075267 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f003d-aa22-4ff5-a619-c9b25b99b0db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:01.076729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f003d-aa22-4ff5-a619-c9b25b99b0db None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:01.101211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-651f003d-aa22-4ff5-a619-c9b25b99b0db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:01.101211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:01.101211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:01.101211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:01.101211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:01.101211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:01.101211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:01.101211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:01.101211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:01.101211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:01.101211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:01.101211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:01.101211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:01.101211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:01.101211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:01.101211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:01.102185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:01.102185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:01.102185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:01.102185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:01.102185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:01.102185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:01.102185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:01.102185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:01.102185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:01.102185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:01.102185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:01.102185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:01.102185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:01.102185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:01.102185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:01.102185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:01.102185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:01.103072 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:01.103072 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:01.103072 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:01.103072 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 235/488] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:01.108253 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f003d-aa22-4ff5-a619-c9b25b99b0db None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:01.188189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 254/489] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:45:01.260092 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f003d-aa22-4ff5-a619-c9b25b99b0db None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:01.269831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdaa0e2-e288-4df2-be2f-488a23233b19 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:01.340328 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9fc877-9892-4737-baaf-b39621c3f40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:01.340965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fc877-9892-4737-baaf-b39621c3f40a None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.341244 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fc877-9892-4737-baaf-b39621c3f40a None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.341515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9fc877-9892-4737-baaf-b39621c3f40a None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.342187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9fc877-9892-4737-baaf-b39621c3f40a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:01.342493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9fc877-9892-4737-baaf-b39621c3f40a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:01.343982 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9fc877-9892-4737-baaf-b39621c3f40a None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:01.366567 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 236/490] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:01.368287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9fc877-9892-4737-baaf-b39621c3f40a None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:01.401946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:01.488227 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9fc877-9892-4737-baaf-b39621c3f40a None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:01.495697 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdaa0e2-e288-4df2-be2f-488a23233b19 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:01.515934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:01.540758 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0c69f-60ed-4e5a-b015-a26d5aabad61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:01.541394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0c69f-60ed-4e5a-b015-a26d5aabad61 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.541670 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0c69f-60ed-4e5a-b015-a26d5aabad61 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.541936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0c69f-60ed-4e5a-b015-a26d5aabad61 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.550558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e1256-4738-4f45-803d-45bd72b1edef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:01.551224 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e1256-4738-4f45-803d-45bd72b1edef None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.551503 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e1256-4738-4f45-803d-45bd72b1edef None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.551614 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0c69f-60ed-4e5a-b015-a26d5aabad61 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:01.551769 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e1256-4738-4f45-803d-45bd72b1edef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.553049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0c69f-60ed-4e5a-b015-a26d5aabad61 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:01.562114 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e1256-4738-4f45-803d-45bd72b1edef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:01.567318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-aec0c69f-60ed-4e5a-b015-a26d5aabad61 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:01.567318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:01.567318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:01.567318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:01.567318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:01.567318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:01.567318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:01.567318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:01.567318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:01.567318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:01.567318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:01.567318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:01.567318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:01.567318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:01.567318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:01.567318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:01.568179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:01.569090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:01.569090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:01.569090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:01.569090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:01.569090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:01.569090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:01.569090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:01.569090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:01.569090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:01.569090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:01.569626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-033e1256-4738-4f45-803d-45bd72b1edef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:01.569626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:01.569626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:01.569626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:01.569626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:01.569626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:01.569626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:01.569626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:01.569626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:01.569626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:01.569626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:01.569626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:01.569626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:01.569626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:01.569626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:01.569626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:01.570434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:01.570434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:01.570434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:01.570434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:01.570434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:01.570434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:01.570434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:01.570434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:01.570434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:01.570434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:01.570434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:01.570434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:01.570434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:01.570434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:01.570434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:01.570434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:01.575680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:01.575680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:01.575680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:01.575680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:01.575680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:01.575680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:01.575680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:01.575680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:01.575680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:01.575680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:01.575680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:01.575680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:01.575680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 255/491] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:01.575680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c602a5-41f1-4189-b6c5-7d3312d998a8 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.576450 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c602a5-41f1-4189-b6c5-7d3312d998a8 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.576450 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c602a5-41f1-4189-b6c5-7d3312d998a8 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.576450 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 256/492] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:01.576450 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c602a5-41f1-4189-b6c5-7d3312d998a8 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:01.576861 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 237/493] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:01.582504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0c69f-60ed-4e5a-b015-a26d5aabad61 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:01.693392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c602a5-41f1-4189-b6c5-7d3312d998a8 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:01.696675 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:01.708331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0c69f-60ed-4e5a-b015-a26d5aabad61 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:01.731145 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574957ba-9e8e-4464-b136-cba3d6f3f1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:01.731731 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574957ba-9e8e-4464-b136-cba3d6f3f1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.732040 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574957ba-9e8e-4464-b136-cba3d6f3f1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.732314 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574957ba-9e8e-4464-b136-cba3d6f3f1a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.746176 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574957ba-9e8e-4464-b136-cba3d6f3f1a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:01.750620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339298d-9b62-4838-b42c-f6a6c5b50e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:01.751051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-574957ba-9e8e-4464-b136-cba3d6f3f1a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:01.751051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:01.751051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:01.751051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:01.751051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:01.751051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:01.751051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:01.751051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:01.751051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:01.751051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:01.751051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:01.751051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:01.751051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:01.751051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:01.751051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:01.751051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:01.752135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:01.752135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:01.752135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:01.752135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:01.752135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:01.752135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:01.752135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:01.752135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:01.752135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:01.752135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:01.752135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:01.752135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:01.752135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:01.752135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:01.752135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:01.752135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:01.755968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:01.755968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:01.755968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:01.755968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:01.755968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:01.755968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:01.755968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:01.755968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:01.755968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:01.755968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:01.755968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:01.755968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:01.756869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339298d-9b62-4838-b42c-f6a6c5b50e78 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.756869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339298d-9b62-4838-b42c-f6a6c5b50e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.756869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339298d-9b62-4838-b42c-f6a6c5b50e78 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.756869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339298d-9b62-4838-b42c-f6a6c5b50e78 None admin] RBAC: Adding query filter params (name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:01.757119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 257/494] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:01.757201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339298d-9b62-4838-b42c-f6a6c5b50e78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:01.757279 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574957ba-9e8e-4464-b136-cba3d6f3f1a4 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:01.757357 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339298d-9b62-4838-b42c-f6a6c5b50e78 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:01.764655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 238/495] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:01.765861 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1646b6-c617-4b67-9b72-053ff1f56a8e None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.766141 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1646b6-c617-4b67-9b72-053ff1f56a8e None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.766407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1646b6-c617-4b67-9b72-053ff1f56a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.785087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-2a1646b6-c617-4b67-9b72-053ff1f56a8e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:01.786278 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2a1646b6-c617-4b67-9b72-053ff1f56a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:01.795519 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:01.859828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 239/496] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:01.861052 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1646b6-c617-4b67-9b72-053ff1f56a8e None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:01.892485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574957ba-9e8e-4464-b136-cba3d6f3f1a4 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:01.943790 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f0636-8e80-4995-a416-3bc7a863b525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:01.945364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f0636-8e80-4995-a416-3bc7a863b525 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:01.945882 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f0636-8e80-4995-a416-3bc7a863b525 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:01.946642 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f0636-8e80-4995-a416-3bc7a863b525 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:01.947658 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f0636-8e80-4995-a416-3bc7a863b525 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:01.948411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f0636-8e80-4995-a416-3bc7a863b525 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:01.950788 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f0636-8e80-4995-a416-3bc7a863b525 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:01.975965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:01.997061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1646b6-c617-4b67-9b72-053ff1f56a8e None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.007112 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 258/497] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:02.008795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f0636-8e80-4995-a416-3bc7a863b525 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.034231 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cf6a7-981c-4cc0-aa6e-825c9b208dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.034956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cf6a7-981c-4cc0-aa6e-825c9b208dba None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.035298 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cf6a7-981c-4cc0-aa6e-825c9b208dba None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.035633 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cf6a7-981c-4cc0-aa6e-825c9b208dba None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.036361 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cf6a7-981c-4cc0-aa6e-825c9b208dba None admin] RBAC: Adding query filter params (name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:02.036724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cf6a7-981c-4cc0-aa6e-825c9b208dba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.038684 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cf6a7-981c-4cc0-aa6e-825c9b208dba None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.048465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 240/498] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:02.049590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cf6a7-981c-4cc0-aa6e-825c9b208dba None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.135111 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:02.150277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f0636-8e80-4995-a416-3bc7a863b525 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.169202 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cf6a7-981c-4cc0-aa6e-825c9b208dba None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.190996 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5326621-f4be-4501-9654-4439efb0bebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.191629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5326621-f4be-4501-9654-4439efb0bebe None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.191904 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5326621-f4be-4501-9654-4439efb0bebe None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.192170 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5326621-f4be-4501-9654-4439efb0bebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.205505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5326621-f4be-4501-9654-4439efb0bebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.206195 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbf532-bcaf-4a40-9401-e26f1ec10ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.206830 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbf532-bcaf-4a40-9401-e26f1ec10ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.207084 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5326621-f4be-4501-9654-4439efb0bebe None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.207181 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbf532-bcaf-4a40-9401-e26f1ec10ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.207368 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbf532-bcaf-4a40-9401-e26f1ec10ef6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.222102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:02.225723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-f5326621-f4be-4501-9654-4439efb0bebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:02.225723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:02.225723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:02.225723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:02.225723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:02.225723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:02.225723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:02.225723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:02.225723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:02.225723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:02.225723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:02.225723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:02.225723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:02.225723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:02.225723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:02.225723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:02.226629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:02.226629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:02.226629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:02.226629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:02.226629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:02.226629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:02.226629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:02.226629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:02.226629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:02.226629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:02.226629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:02.226629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:02.226629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:02.226629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:02.226629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:02.226629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:02.226629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:02.232136 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:02.232136 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:02.232136 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:02.232136 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 259/499] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:02.232136 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5326621-f4be-4501-9654-4439efb0bebe None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.282795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbf532-bcaf-4a40-9401-e26f1ec10ef6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.284196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fbf532-bcaf-4a40-9401-e26f1ec10ef6 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.332159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5326621-f4be-4501-9654-4439efb0bebe None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.367729 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe4a6f-7c86-4f2b-acea-588b2188f880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.368364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe4a6f-7c86-4f2b-acea-588b2188f880 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.368643 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe4a6f-7c86-4f2b-acea-588b2188f880 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.368915 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe4a6f-7c86-4f2b-acea-588b2188f880 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.380680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe4a6f-7c86-4f2b-acea-588b2188f880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.382140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe4a6f-7c86-4f2b-acea-588b2188f880 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.383908 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:02.388385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 241/500] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:02.389453 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbf532-bcaf-4a40-9401-e26f1ec10ef6 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.397716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-eabe4a6f-7c86-4f2b-acea-588b2188f880 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:02.397716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:02.397716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:02.397716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:02.397716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:02.397716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:02.397716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:02.397716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:02.397716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:02.397716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:02.397716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:02.397716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:02.397716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:02.397716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:02.397716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:02.397716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:02.398685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:02.398685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:02.398685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:02.398685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:02.398685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:02.398685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:02.398685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:02.398685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:02.398685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:02.398685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:02.398685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:02.398685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:02.398685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:02.398685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:02.398685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:02.398685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:02.398685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:02.400198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:02.400198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:02.400198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:02.400198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 260/501] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:02.400198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe4a6f-7c86-4f2b-acea-588b2188f880 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.440695 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:02.496135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbf532-bcaf-4a40-9401-e26f1ec10ef6 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.503227 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe4a6f-7c86-4f2b-acea-588b2188f880 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.530438 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bea880-5885-4419-886b-ee4e816da3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.531103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bea880-5885-4419-886b-ee4e816da3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.531380 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bea880-5885-4419-886b-ee4e816da3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.531655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bea880-5885-4419-886b-ee4e816da3a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.536865 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847e1df-9017-484e-9901-a36228380c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.537412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847e1df-9017-484e-9901-a36228380c88 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.537692 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847e1df-9017-484e-9901-a36228380c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.537963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847e1df-9017-484e-9901-a36228380c88 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.538597 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847e1df-9017-484e-9901-a36228380c88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:02.538934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847e1df-9017-484e-9901-a36228380c88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.540287 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2847e1df-9017-484e-9901-a36228380c88 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.542211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bea880-5885-4419-886b-ee4e816da3a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:02.547071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-c5bea880-5885-4419-886b-ee4e816da3a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:02.547071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:02.547071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:02.547071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:02.547071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:02.547071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:02.547071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:02.547071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:02.547071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:02.547071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:02.547071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:02.547071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:02.547071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:02.547071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:02.547071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:02.547071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:02.548587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:02.548587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:02.548587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:02.548587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:02.548587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:02.548587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:02.548587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:02.548587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:02.548587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:02.548587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:02.548587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:02.548587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:02.548587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:02.548587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:02.548587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:02.548587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:02.549370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:02.549370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:02.549370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:02.549370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:02.549370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:02.549370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:02.549370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:02.549370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:02.549370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:02.549370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:02.549370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:02.549370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:02.549370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 242/502] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:02.550110 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76be37a-3b9d-4774-b0fe-b66d0d3e617f None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.550175 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76be37a-3b9d-4774-b0fe-b66d0d3e617f None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.550458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76be37a-3b9d-4774-b0fe-b66d0d3e617f None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.551782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 243/503] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:02.552356 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76be37a-3b9d-4774-b0fe-b66d0d3e617f None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.558694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 261/504] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:02.559563 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847e1df-9017-484e-9901-a36228380c88 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.614813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:02.656294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76be37a-3b9d-4774-b0fe-b66d0d3e617f None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.659252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:02.659698 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847e1df-9017-484e-9901-a36228380c88 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.690434 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b99c8-f22d-4a74-a157-27011ebde134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.691095 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b99c8-f22d-4a74-a157-27011ebde134 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.691374 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b99c8-f22d-4a74-a157-27011ebde134 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.691649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b99c8-f22d-4a74-a157-27011ebde134 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.694763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40338af7-a101-4776-b39b-d57bf80b1edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.695381 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40338af7-a101-4776-b39b-d57bf80b1edd None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.695662 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40338af7-a101-4776-b39b-d57bf80b1edd None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.695933 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40338af7-a101-4776-b39b-d57bf80b1edd None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.696568 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40338af7-a101-4776-b39b-d57bf80b1edd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:02.696872 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40338af7-a101-4776-b39b-d57bf80b1edd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.698243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40338af7-a101-4776-b39b-d57bf80b1edd None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.701659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b99c8-f22d-4a74-a157-27011ebde134 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.703139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b99c8-f22d-4a74-a157-27011ebde134 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.709805 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 262/505] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:02.710612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40338af7-a101-4776-b39b-d57bf80b1edd None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.723285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-946b99c8-f22d-4a74-a157-27011ebde134 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:02.723285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:02.723285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:02.723285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:02.723285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:02.723285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:02.723285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:02.723285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:02.723285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:02.723285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:02.723285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:02.723285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:02.723285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:02.723285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:02.723285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:02.723285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:02.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:02.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:02.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:02.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:02.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:02.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:02.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:02.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:02.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:02.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:02.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:02.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:02.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:02.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:02.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:02.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:02.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:02.725610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:02.725610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:02.725610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:02.725610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 244/506] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:02.725894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd99ab-14ed-4bd8-bb92-3462201aebb6 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.726168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd99ab-14ed-4bd8-bb92-3462201aebb6 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.726441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd99ab-14ed-4bd8-bb92-3462201aebb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.743781 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-73dd99ab-14ed-4bd8-bb92-3462201aebb6 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:02.744971 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-73dd99ab-14ed-4bd8-bb92-3462201aebb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:02.810869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 245/507] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:02.811794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd99ab-14ed-4bd8-bb92-3462201aebb6 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.815451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40338af7-a101-4776-b39b-d57bf80b1edd None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.829539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:02.852379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e8dee-37fd-4f0c-ac97-46e2bd9470dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.852985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e8dee-37fd-4f0c-ac97-46e2bd9470dc None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.853269 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e8dee-37fd-4f0c-ac97-46e2bd9470dc None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.853541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e8dee-37fd-4f0c-ac97-46e2bd9470dc None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.854200 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e8dee-37fd-4f0c-ac97-46e2bd9470dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:02.854618 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e8dee-37fd-4f0c-ac97-46e2bd9470dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.856127 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e8dee-37fd-4f0c-ac97-46e2bd9470dc None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.865424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 263/508] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:02.866548 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e8dee-37fd-4f0c-ac97-46e2bd9470dc None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:02.914872 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:02.931707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd99ab-14ed-4bd8-bb92-3462201aebb6 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.970263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d490111-60be-4d7b-990f-aa529bdb8c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:02.970930 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490111-60be-4d7b-990f-aa529bdb8c79 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:02.971210 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490111-60be-4d7b-990f-aa529bdb8c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:02.971742 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490111-60be-4d7b-990f-aa529bdb8c79 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:02.975503 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e8dee-37fd-4f0c-ac97-46e2bd9470dc None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:02.979259 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d490111-60be-4d7b-990f-aa529bdb8c79 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:02.980711 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d490111-60be-4d7b-990f-aa529bdb8c79 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:02.993929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-7d490111-60be-4d7b-990f-aa529bdb8c79 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 14:45:02.993929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:02.993929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:02.993929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:02.993929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:02.993929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:02.993929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:02.993929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:02.993929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:02.993929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:02.993929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:02.993929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:02.993929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:02.993929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:02.993929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:02.993929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:02.994908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:03.001013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:03.001013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:03.001013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:03.001013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:03.001013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:03.001013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:03.001013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:03.001013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:03.001013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 14:45:03.001013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:03.001013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 246/509] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:03.001013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d490111-60be-4d7b-990f-aa529bdb8c79 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.015706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da019313-a31c-4120-8aeb-2e58e7f45f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.016338 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da019313-a31c-4120-8aeb-2e58e7f45f2d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.016640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da019313-a31c-4120-8aeb-2e58e7f45f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.016924 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da019313-a31c-4120-8aeb-2e58e7f45f2d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.024892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da019313-a31c-4120-8aeb-2e58e7f45f2d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.026335 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da019313-a31c-4120-8aeb-2e58e7f45f2d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.039219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-da019313-a31c-4120-8aeb-2e58e7f45f2d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 14:45:03.039219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:03.039219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:03.039219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:03.039219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:03.039219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:03.039219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:03.039219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:03.039219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:03.039219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:03.039219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:03.039219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:03.039219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:03.039219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:03.039219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:03.039219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:03.040161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:03.041044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:03.041044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:03.041044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:03.041044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:03.041044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:03.041044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:03.041044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:03.041044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:03.041044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 14:45:03.041044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:03.044918 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 264/510] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:03.045616 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da019313-a31c-4120-8aeb-2e58e7f45f2d None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.080338 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:03.105860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d490111-60be-4d7b-990f-aa529bdb8c79 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.140781 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8989c-8bae-4e66-a399-2bab5b64f1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.141386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8989c-8bae-4e66-a399-2bab5b64f1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.142227 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8989c-8bae-4e66-a399-2bab5b64f1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.142495 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8989c-8bae-4e66-a399-2bab5b64f1e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.143181 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8989c-8bae-4e66-a399-2bab5b64f1e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:03.143461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8989c-8bae-4e66-a399-2bab5b64f1e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.144851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8989c-8bae-4e66-a399-2bab5b64f1e8 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.147404 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da019313-a31c-4120-8aeb-2e58e7f45f2d None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.156921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 247/511] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:03.157733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8989c-8bae-4e66-a399-2bab5b64f1e8 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.165964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:03.184489 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430d50d-2d4a-4c4a-913d-575e8bbcb929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.185122 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430d50d-2d4a-4c4a-913d-575e8bbcb929 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.185403 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430d50d-2d4a-4c4a-913d-575e8bbcb929 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.185674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430d50d-2d4a-4c4a-913d-575e8bbcb929 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.196989 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6430d50d-2d4a-4c4a-913d-575e8bbcb929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.198445 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6430d50d-2d4a-4c4a-913d-575e8bbcb929 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.213680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-6430d50d-2d4a-4c4a-913d-575e8bbcb929 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:03.213680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:03.213680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:03.213680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:03.213680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:03.213680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:03.213680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:03.213680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:03.213680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:03.213680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:03.213680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:03.213680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:03.213680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:03.213680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:03.213680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:03.213680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:03.214707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:03.214707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:03.214707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:03.214707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:03.214707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:03.214707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:03.214707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:03.214707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:03.214707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:03.214707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:03.214707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:03.214707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:03.214707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:03.214707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:03.214707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:03.214707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:03.214707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:03.220473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:03.220473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:03.220473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:03.220473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 265/512] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:03.220473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430d50d-2d4a-4c4a-913d-575e8bbcb929 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.259679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8989c-8bae-4e66-a399-2bab5b64f1e8 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.293370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8b7f1-09f2-41c5-8d25-808174f57202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.293983 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8b7f1-09f2-41c5-8d25-808174f57202 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.294257 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8b7f1-09f2-41c5-8d25-808174f57202 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.294532 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8b7f1-09f2-41c5-8d25-808174f57202 None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.295284 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8b7f1-09f2-41c5-8d25-808174f57202 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:03.295585 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8b7f1-09f2-41c5-8d25-808174f57202 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.297141 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8b7f1-09f2-41c5-8d25-808174f57202 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.303350 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:03.320846 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430d50d-2d4a-4c4a-913d-575e8bbcb929 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.335236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 248/513] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:03.336034 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8b7f1-09f2-41c5-8d25-808174f57202 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.360505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4b43b-08a8-4f1f-8463-4b31dd5efd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.361115 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4b43b-08a8-4f1f-8463-4b31dd5efd65 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.361389 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4b43b-08a8-4f1f-8463-4b31dd5efd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.361666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4b43b-08a8-4f1f-8463-4b31dd5efd65 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.362392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4b43b-08a8-4f1f-8463-4b31dd5efd65 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:03.362695 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4b43b-08a8-4f1f-8463-4b31dd5efd65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.364164 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4b43b-08a8-4f1f-8463-4b31dd5efd65 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.381492 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 266/514] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:03.382368 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4b43b-08a8-4f1f-8463-4b31dd5efd65 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.402610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:03.440424 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8b7f1-09f2-41c5-8d25-808174f57202 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.474957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e980ad4-9fe2-47b5-8091-e5fb96dcdd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.475581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e980ad4-9fe2-47b5-8091-e5fb96dcdd5f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.475853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e980ad4-9fe2-47b5-8091-e5fb96dcdd5f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.476125 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e980ad4-9fe2-47b5-8091-e5fb96dcdd5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.481560 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4b43b-08a8-4f1f-8463-4b31dd5efd65 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.485954 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e980ad4-9fe2-47b5-8091-e5fb96dcdd5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.487451 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e980ad4-9fe2-47b5-8091-e5fb96dcdd5f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.501778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-8e980ad4-9fe2-47b5-8091-e5fb96dcdd5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:03.501778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:03.501778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:03.501778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:03.501778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:03.501778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:03.501778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:03.501778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:03.501778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:03.501778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:03.501778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:03.501778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:03.501778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:03.501778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:03.501778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:03.501778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:03.502917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:03.502917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:03.502917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:03.502917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:03.502917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:03.502917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:03.502917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:03.502917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:03.502917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:03.502917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:03.502917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:03.502917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:03.502917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:03.502917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:03.502917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:03.502917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:03.502917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:03.503794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:03.503794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:03.503794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:03.503794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 249/515] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:03.504166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e980ad4-9fe2-47b5-8091-e5fb96dcdd5f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.519434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e5875-4012-4d82-abe3-328d7d21e0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.520103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e5875-4012-4d82-abe3-328d7d21e0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.520381 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e5875-4012-4d82-abe3-328d7d21e0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.520653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e5875-4012-4d82-abe3-328d7d21e0c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.527415 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e5875-4012-4d82-abe3-328d7d21e0c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.528956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e5875-4012-4d82-abe3-328d7d21e0c0 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.541485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-bb2e5875-4012-4d82-abe3-328d7d21e0c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:03.541485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:03.541485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:03.541485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:03.541485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:03.541485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:03.541485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:03.541485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:03.541485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:03.541485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:03.541485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:03.541485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:03.541485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:03.541485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:03.541485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:03.542339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:03.542339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:03.542339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:03.542339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:03.542339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:03.542339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:03.542339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:03.542339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:03.542339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:03.542339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:03.542339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:03.542339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:03.542339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:03.542339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:03.542339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:03.542339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:03.542339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:03.543218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:03.543218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:03.543218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:03.543218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:03.543218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:03.543218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:03.543218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:03.543218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:03.543218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 267/516] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:03.543678 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e5875-4012-4d82-abe3-328d7d21e0c0 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.552496 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:03.607347 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e980ad4-9fe2-47b5-8091-e5fb96dcdd5f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.623781 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:03.642147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8a37d-9ee1-40fe-ba17-087d2f1d77c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.642371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e5875-4012-4d82-abe3-328d7d21e0c0 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.642775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8a37d-9ee1-40fe-ba17-087d2f1d77c9 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.643057 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8a37d-9ee1-40fe-ba17-087d2f1d77c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.643324 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8a37d-9ee1-40fe-ba17-087d2f1d77c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.643958 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8a37d-9ee1-40fe-ba17-087d2f1d77c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:03.644256 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8a37d-9ee1-40fe-ba17-087d2f1d77c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.645643 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8a37d-9ee1-40fe-ba17-087d2f1d77c9 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.659158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 250/517] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:03.659965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8a37d-9ee1-40fe-ba17-087d2f1d77c9 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.677622 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7caab-38f5-4284-b2ce-2b8c6a5d2df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.678231 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7caab-38f5-4284-b2ce-2b8c6a5d2df6 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.678531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7caab-38f5-4284-b2ce-2b8c6a5d2df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.678832 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7caab-38f5-4284-b2ce-2b8c6a5d2df6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.745897 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7caab-38f5-4284-b2ce-2b8c6a5d2df6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.747411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7caab-38f5-4284-b2ce-2b8c6a5d2df6 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.755504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:03.761094 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8a37d-9ee1-40fe-ba17-087d2f1d77c9 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.795305 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258f213-87d3-4356-8f92-04d2b716b8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.795913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258f213-87d3-4356-8f92-04d2b716b8bd None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.796182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258f213-87d3-4356-8f92-04d2b716b8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.796446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258f213-87d3-4356-8f92-04d2b716b8bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.840467 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:03.848980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 268/518] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:03.849831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7caab-38f5-4284-b2ce-2b8c6a5d2df6 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.868048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4258f213-87d3-4356-8f92-04d2b716b8bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.869453 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4258f213-87d3-4356-8f92-04d2b716b8bd None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:03.952359 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7caab-38f5-4284-b2ce-2b8c6a5d2df6 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:03.960542 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:03.974171 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 251/519] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:03.975274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258f213-87d3-4356-8f92-04d2b716b8bd None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:03.992217 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9604c-b35f-4948-a84d-4bb4dffb751b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:03.992812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9604c-b35f-4948-a84d-4bb4dffb751b None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:03.993102 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9604c-b35f-4948-a84d-4bb4dffb751b None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:03.993348 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9604c-b35f-4948-a84d-4bb4dffb751b None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:03.993982 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9604c-b35f-4948-a84d-4bb4dffb751b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:03.994279 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9604c-b35f-4948-a84d-4bb4dffb751b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:03.995677 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9604c-b35f-4948-a84d-4bb4dffb751b None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.007384 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 269/520] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:04.008203 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9604c-b35f-4948-a84d-4bb4dffb751b None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.054994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:04.078202 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258f213-87d3-4356-8f92-04d2b716b8bd None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.109226 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9604c-b35f-4948-a84d-4bb4dffb751b None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.112451 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe02ae-b2f3-453b-8954-63bf6f98c0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.113062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe02ae-b2f3-453b-8954-63bf6f98c0b4 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.113339 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe02ae-b2f3-453b-8954-63bf6f98c0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.113612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe02ae-b2f3-453b-8954-63bf6f98c0b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.114326 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe02ae-b2f3-453b-8954-63bf6f98c0b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:04.114637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe02ae-b2f3-453b-8954-63bf6f98c0b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.116143 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe02ae-b2f3-453b-8954-63bf6f98c0b4 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.133917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 252/521] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:04.134679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe02ae-b2f3-453b-8954-63bf6f98c0b4 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.148761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94357aab-7be3-4d02-91e2-a93517d29c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.149388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94357aab-7be3-4d02-91e2-a93517d29c14 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.149669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94357aab-7be3-4d02-91e2-a93517d29c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.149939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94357aab-7be3-4d02-91e2-a93517d29c14 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.157469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94357aab-7be3-4d02-91e2-a93517d29c14 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.158927 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94357aab-7be3-4d02-91e2-a93517d29c14 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.169970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-94357aab-7be3-4d02-91e2-a93517d29c14 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 14:45:04.169970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:04.169970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:04.169970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:04.169970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:04.169970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:04.169970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:04.169970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:04.169970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:04.169970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:04.169970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:04.169970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:04.169970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:04.169970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:04.169970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:04.169970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:04.171033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:04.171943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:04.171943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:04.171943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:04.171943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:04.171943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:04.171943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:04.171943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:04.171943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:04.171943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 14:45:04.171943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:04.171943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 270/522] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:04.171943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94357aab-7be3-4d02-91e2-a93517d29c14 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.203105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:04.236943 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe02ae-b2f3-453b-8954-63bf6f98c0b4 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.272173 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009ef9b-d5b0-49e4-8a91-ca3dae0db52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.272798 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009ef9b-d5b0-49e4-8a91-ca3dae0db52f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.273075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009ef9b-d5b0-49e4-8a91-ca3dae0db52f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.273339 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2009ef9b-d5b0-49e4-8a91-ca3dae0db52f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.275947 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:04.276664 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94357aab-7be3-4d02-91e2-a93517d29c14 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.284573 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2009ef9b-d5b0-49e4-8a91-ca3dae0db52f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.286064 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2009ef9b-d5b0-49e4-8a91-ca3dae0db52f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.300228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-2009ef9b-d5b0-49e4-8a91-ca3dae0db52f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:04.300228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:04.300228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:04.300228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:04.300228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:04.300228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:04.300228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:04.300228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:04.300228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:04.300228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:04.300228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:04.300228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:04.300228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:04.300228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:04.300228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:04.300228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:04.301165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:04.301165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:04.301165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:04.301165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:04.301165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:04.301165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:04.301165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:04.301165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:04.301165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:04.301165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:04.301165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:04.301165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:04.301165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:04.301165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:04.301165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:04.301165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:04.301165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:04.302014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:04.302014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:04.302014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:04.302014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 253/523] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:04.302014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009ef9b-d5b0-49e4-8a91-ca3dae0db52f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.314569 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d408b1-b992-473e-a00e-3b22bb32f810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.315250 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d408b1-b992-473e-a00e-3b22bb32f810 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.315587 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d408b1-b992-473e-a00e-3b22bb32f810 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.315862 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d408b1-b992-473e-a00e-3b22bb32f810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.327180 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d408b1-b992-473e-a00e-3b22bb32f810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.328139 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d408b1-b992-473e-a00e-3b22bb32f810 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.342829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-97d408b1-b992-473e-a00e-3b22bb32f810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:04.342829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:04.342829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:04.342829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:04.342829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:04.342829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:04.342829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:04.342829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:04.342829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:04.342829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:04.342829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:04.342829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:04.342829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:04.342829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:04.342829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:04.342829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:04.349392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:04.349392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:04.349392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:04.349392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:04.349392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:04.349392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:04.349392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:04.349392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:04.349392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:04.349392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:04.349392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:04.349392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:04.349392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:04.349392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:04.349392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:04.349392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:04.349392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:04.350263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:04.350263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:04.350263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:04.350263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 271/524] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:04.350263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d408b1-b992-473e-a00e-3b22bb32f810 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.411063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2009ef9b-d5b0-49e4-8a91-ca3dae0db52f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.428820 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:04.445907 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268714b9-a77b-424f-9ac4-3ad67b1af563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.446525 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268714b9-a77b-424f-9ac4-3ad67b1af563 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.446827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268714b9-a77b-424f-9ac4-3ad67b1af563 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.447103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268714b9-a77b-424f-9ac4-3ad67b1af563 None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.447738 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268714b9-a77b-424f-9ac4-3ad67b1af563 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.449203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268714b9-a77b-424f-9ac4-3ad67b1af563 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.450993 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d408b1-b992-473e-a00e-3b22bb32f810 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.470614 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 254/525] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:45:04 2022] POST /identity/v3/users => generated 247 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:04.471289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268714b9-a77b-424f-9ac4-3ad67b1af563 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.500120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c0c20-bd2c-413a-9fba-8f7766acf501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.500764 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c0c20-bd2c-413a-9fba-8f7766acf501 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.501085 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c0c20-bd2c-413a-9fba-8f7766acf501 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.501363 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c0c20-bd2c-413a-9fba-8f7766acf501 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.502115 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c0c20-bd2c-413a-9fba-8f7766acf501 None admin] RBAC: Adding query filter params (role.id=35bdd5122aae45ebbe44158a433d2f99, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=0c9be41c39494e54a14398a1f03c4d69) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:04.502440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c0c20-bd2c-413a-9fba-8f7766acf501 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.504118 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c0c20-bd2c-413a-9fba-8f7766acf501 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.522766 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 272/526] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/role_assignments?user.id=0c9be41c39494e54a14398a1f03c4d69&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=35bdd5122aae45ebbe44158a433d2f99 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:04.522766 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c0c20-bd2c-413a-9fba-8f7766acf501 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.545950 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:04.636172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268714b9-a77b-424f-9ac4-3ad67b1af563 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.655008 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c0c20-bd2c-413a-9fba-8f7766acf501 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.696438 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570caab-947f-47be-b153-ab68030b5c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.696994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570caab-947f-47be-b153-ab68030b5c05 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.697274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570caab-947f-47be-b153-ab68030b5c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.697544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570caab-947f-47be-b153-ab68030b5c05 None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.698390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570caab-947f-47be-b153-ab68030b5c05 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:04.698592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570caab-947f-47be-b153-ab68030b5c05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.700097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570caab-947f-47be-b153-ab68030b5c05 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.706324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c2633-275c-44eb-a240-0504c8c6b6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.707027 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c2633-275c-44eb-a240-0504c8c6b6d8 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.707308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c2633-275c-44eb-a240-0504c8c6b6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.707578 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c2633-275c-44eb-a240-0504c8c6b6d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.708237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c2633-275c-44eb-a240-0504c8c6b6d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:04.708538 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c2633-275c-44eb-a240-0504c8c6b6d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.709944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c2633-275c-44eb-a240-0504c8c6b6d8 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.725003 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 255/527] 10.222.0.26 () {56 vars in 1139 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:04.725003 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570caab-947f-47be-b153-ab68030b5c05 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.726745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 273/528] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:04.729546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c2633-275c-44eb-a240-0504c8c6b6d8 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.732433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:04.849695 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570caab-947f-47be-b153-ab68030b5c05 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.851447 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:04.857736 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c2633-275c-44eb-a240-0504c8c6b6d8 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:04.891062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f259327-7833-465d-a025-f8c3c200ac76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.892382 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f259327-7833-465d-a025-f8c3c200ac76 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.892800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f259327-7833-465d-a025-f8c3c200ac76 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.893169 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f259327-7833-465d-a025-f8c3c200ac76 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.893881 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f259327-7833-465d-a025-f8c3c200ac76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:04.894251 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f259327-7833-465d-a025-f8c3c200ac76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.895926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f259327-7833-465d-a025-f8c3c200ac76 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.901243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c3ead-7024-426b-a053-4ce365e50e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:04.901850 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c3ead-7024-426b-a053-4ce365e50e9e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:04.902179 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c3ead-7024-426b-a053-4ce365e50e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:04.902454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c3ead-7024-426b-a053-4ce365e50e9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:04.911022 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 256/529] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:04.912092 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f259327-7833-465d-a025-f8c3c200ac76 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:04.983236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c3ead-7024-426b-a053-4ce365e50e9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:04.984893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c3ead-7024-426b-a053-4ce365e50e9e None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:04.987509 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:05.031661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f259327-7833-465d-a025-f8c3c200ac76 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:05.071511 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe126a8-e78f-4ad0-830b-547f3e650cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:05.072132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe126a8-e78f-4ad0-830b-547f3e650cdc None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:05.072412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe126a8-e78f-4ad0-830b-547f3e650cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:05.072680 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe126a8-e78f-4ad0-830b-547f3e650cdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:05.080355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe126a8-e78f-4ad0-830b-547f3e650cdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:05.081820 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe126a8-e78f-4ad0-830b-547f3e650cdc None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.094024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-9fe126a8-e78f-4ad0-830b-547f3e650cdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:05.094024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:05.094024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:05.094024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:05.094024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:05.094024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:05.094024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:05.094024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:05.094024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:05.094024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:05.094024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:05.094024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:05.094024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:05.094024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:05.094024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:05.095733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:05.095733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:05.095733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:05.095733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:05.095733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:05.095733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:05.095733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:05.095733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:05.095733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:05.095733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:05.095733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:05.095733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:05.095733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:05.095733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:05.095733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:05.095733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:05.095733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:05.097100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:05.097100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:05.097100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:05.097100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:05.097100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:05.097100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:05.097100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:05.097100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:05.097100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 257/530] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:05.097100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe126a8-e78f-4ad0-830b-547f3e650cdc None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:05.118331 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 274/531] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:05.119238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c3ead-7024-426b-a053-4ce365e50e9e None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:05.127473 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:05.207641 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe126a8-e78f-4ad0-830b-547f3e650cdc None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:05.234849 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c3ead-7024-426b-a053-4ce365e50e9e None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:05.239060 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:05.244217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc9c8f-13d8-4631-9d57-c7918e7f98d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:05.244992 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc9c8f-13d8-4631-9d57-c7918e7f98d3 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:05.245367 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc9c8f-13d8-4631-9d57-c7918e7f98d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:05.245737 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc9c8f-13d8-4631-9d57-c7918e7f98d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:05.254093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc9c8f-13d8-4631-9d57-c7918e7f98d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:05.255926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc9c8f-13d8-4631-9d57-c7918e7f98d3 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.269468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-5bbc9c8f-13d8-4631-9d57-c7918e7f98d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:05.269468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:05.269468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:05.269468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:05.269468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:05.269468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:05.269468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:05.269468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:05.269468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:05.269468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:05.269468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:05.269468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:05.269468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:05.269468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:05.269468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:05.270465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:05.270465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:05.270465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:05.270465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:05.270465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:05.270465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:05.270465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:05.270465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:05.270465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:05.270465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:05.270465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:05.270465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:05.270465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:05.270465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:05.270465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:05.270465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:05.270465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:05.271351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:05.271351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:05.271351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:05.271351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:05.271351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:05.271351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:05.271351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:05.271351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:05.273594 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfab80-8868-4d91-b681-ed4af538957e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:05.274200 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfab80-8868-4d91-b681-ed4af538957e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:05.274479 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfab80-8868-4d91-b681-ed4af538957e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:05.274780 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfab80-8868-4d91-b681-ed4af538957e None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:05.275519 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfab80-8868-4d91-b681-ed4af538957e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:05.276083 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfab80-8868-4d91-b681-ed4af538957e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:05.276197 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 258/532] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:05.277076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc9c8f-13d8-4631-9d57-c7918e7f98d3 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:05.277554 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfab80-8868-4d91-b681-ed4af538957e None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.302827 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 275/533] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:05.303810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfab80-8868-4d91-b681-ed4af538957e None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:05.379420 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:05.400948 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc9c8f-13d8-4631-9d57-c7918e7f98d3 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:05.437315 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfab80-8868-4d91-b681-ed4af538957e None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:05.438205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990ac0e-e5d0-4bef-bba0-0ce92116eb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:05.439615 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990ac0e-e5d0-4bef-bba0-0ce92116eb63 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:05.440090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990ac0e-e5d0-4bef-bba0-0ce92116eb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:05.440588 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990ac0e-e5d0-4bef-bba0-0ce92116eb63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:05.454098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1990ac0e-e5d0-4bef-bba0-0ce92116eb63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:05.455622 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1990ac0e-e5d0-4bef-bba0-0ce92116eb63 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.472669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-1990ac0e-e5d0-4bef-bba0-0ce92116eb63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:05.472669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:05.472669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:05.472669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:05.472669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:05.472669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:05.472669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:05.472669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:05.472669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:05.472669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:05.472669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:05.472669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:05.472669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:05.472669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:05.472669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:05.472669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:05.473646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:05.473646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:05.473646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:05.473646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:05.473646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:05.473646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:05.473646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:05.473646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:05.473646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:05.473646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:05.473646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:05.473646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:05.473646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:05.473646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:05.473646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:05.473646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:05.473646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:05.474485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:05.474485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:05.474485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:05.479006 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 259/534] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:05.479332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcf1e3-941e-4608-8a53-930b471f86a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:05.479960 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcf1e3-941e-4608-8a53-930b471f86a8 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:05.480133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990ac0e-e5d0-4bef-bba0-0ce92116eb63 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:05.480283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcf1e3-941e-4608-8a53-930b471f86a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:05.480487 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcf1e3-941e-4608-8a53-930b471f86a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:05.481190 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dcf1e3-941e-4608-8a53-930b471f86a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:05.481490 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dcf1e3-941e-4608-8a53-930b471f86a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:05.483235 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dcf1e3-941e-4608-8a53-930b471f86a8 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.500681 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 276/535] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:05.501667 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcf1e3-941e-4608-8a53-930b471f86a8 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:05.534386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:05.615260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990ac0e-e5d0-4bef-bba0-0ce92116eb63 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:05.623014 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcf1e3-941e-4608-8a53-930b471f86a8 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:05.658839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e64a9-d52d-4047-8eca-7837e7723b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:05.659460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5c195-1d4a-4a1d-b76d-a4a2cf2614ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:05.659656 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e64a9-d52d-4047-8eca-7837e7723b98 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:05.659961 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e64a9-d52d-4047-8eca-7837e7723b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:05.660102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5c195-1d4a-4a1d-b76d-a4a2cf2614ca None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:05.660321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5c195-1d4a-4a1d-b76d-a4a2cf2614ca None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:05.660546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e64a9-d52d-4047-8eca-7837e7723b98 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:05.660676 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5c195-1d4a-4a1d-b76d-a4a2cf2614ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:05.661181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e64a9-d52d-4047-8eca-7837e7723b98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:05.661330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5c195-1d4a-4a1d-b76d-a4a2cf2614ca None admin] RBAC: Adding query filter params (role.id=35bdd5122aae45ebbe44158a433d2f99, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=bb5248cb1c39412680cce0d9891fe1ff) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:05.661582 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5c195-1d4a-4a1d-b76d-a4a2cf2614ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:05.661732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e64a9-d52d-4047-8eca-7837e7723b98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:05.663025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5c195-1d4a-4a1d-b76d-a4a2cf2614ca None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.663199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e64a9-d52d-4047-8eca-7837e7723b98 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.664884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:05.675793 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 260/536] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:05 2022] GET /identity/v3/role_assignments?user.id=bb5248cb1c39412680cce0d9891fe1ff&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=35bdd5122aae45ebbe44158a433d2f99 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:05.676733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5c195-1d4a-4a1d-b76d-a4a2cf2614ca None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:05.682037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 277/537] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:05.683233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e64a9-d52d-4047-8eca-7837e7723b98 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:05.779971 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:05.803221 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e64a9-d52d-4047-8eca-7837e7723b98 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:05.844880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fa7b0-1242-47f1-ba80-fe90f76c6c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:05.845531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fa7b0-1242-47f1-ba80-fe90f76c6c83 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:05.845809 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fa7b0-1242-47f1-ba80-fe90f76c6c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:05.846080 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fa7b0-1242-47f1-ba80-fe90f76c6c83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:05.864368 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fa7b0-1242-47f1-ba80-fe90f76c6c83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:05.865854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fa7b0-1242-47f1-ba80-fe90f76c6c83 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.888923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5c195-1d4a-4a1d-b76d-a4a2cf2614ca None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:05.891902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-d80fa7b0-1242-47f1-ba80-fe90f76c6c83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:05.891902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:05.891902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:05.891902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:05.891902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:05.891902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:05.891902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:05.891902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:05.891902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:05.891902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:05.891902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:05.891902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:05.891902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:05.891902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:05.891902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:05.892761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:05.892761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:05.892761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:05.892761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:05.892761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:05.892761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:05.892761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:05.892761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:05.892761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:05.892761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:05.892761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:05.892761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:05.892761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:05.892761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:05.892761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:05.892761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:05.892761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:05.893611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:05.893611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:05.893611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:05.893611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:05.893611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:05.893611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:05.893611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:05.893611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:05.896272 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 278/538] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:05.906675 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fa7b0-1242-47f1-ba80-fe90f76c6c83 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:05.966494 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94320b05-54b3-4fc8-8ed4-c4e6d12c98b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:05.967129 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94320b05-54b3-4fc8-8ed4-c4e6d12c98b6 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:05.967405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94320b05-54b3-4fc8-8ed4-c4e6d12c98b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:05.967675 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94320b05-54b3-4fc8-8ed4-c4e6d12c98b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:05.968402 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94320b05-54b3-4fc8-8ed4-c4e6d12c98b6 None admin] RBAC: Adding query filter params (role.id=35bdd5122aae45ebbe44158a433d2f99, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=1d265d111b534944b7812cf067e317a2) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:05.968699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94320b05-54b3-4fc8-8ed4-c4e6d12c98b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:05.970663 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94320b05-54b3-4fc8-8ed4-c4e6d12c98b6 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:05.976980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 261/539] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:05 2022] GET /identity/v3/role_assignments?user.id=1d265d111b534944b7812cf067e317a2&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=35bdd5122aae45ebbe44158a433d2f99 => generated 259 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:06.057042 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fa7b0-1242-47f1-ba80-fe90f76c6c83 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:06.094488 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83666ee6-189f-4cc5-8cb1-d0cc3720a5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:06.095654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83666ee6-189f-4cc5-8cb1-d0cc3720a5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:06.095935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83666ee6-189f-4cc5-8cb1-d0cc3720a5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:06.096207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83666ee6-189f-4cc5-8cb1-d0cc3720a5f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:06.097165 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83666ee6-189f-4cc5-8cb1-d0cc3720a5f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:06.097468 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83666ee6-189f-4cc5-8cb1-d0cc3720a5f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:06.098959 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83666ee6-189f-4cc5-8cb1-d0cc3720a5f7 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:06.113095 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 279/540] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:06.124002 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94320b05-54b3-4fc8-8ed4-c4e6d12c98b6 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:06.220927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:06.276693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94320b05-54b3-4fc8-8ed4-c4e6d12c98b6 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:06.318351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d43cbf-9e53-43a2-8989-5f56b5b1b48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:06.319009 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d43cbf-9e53-43a2-8989-5f56b5b1b48c None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:06.319289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d43cbf-9e53-43a2-8989-5f56b5b1b48c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:06.319561 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d43cbf-9e53-43a2-8989-5f56b5b1b48c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:06.320294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d43cbf-9e53-43a2-8989-5f56b5b1b48c None admin] RBAC: Adding query filter params (role.id=35bdd5122aae45ebbe44158a433d2f99, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=f77bbd8de87042aab3674db366635cf5) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:06.320602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d43cbf-9e53-43a2-8989-5f56b5b1b48c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:06.322044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d43cbf-9e53-43a2-8989-5f56b5b1b48c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:06.328723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 262/541] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:06 2022] GET /identity/v3/role_assignments?user.id=f77bbd8de87042aab3674db366635cf5&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=35bdd5122aae45ebbe44158a433d2f99 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:06.906525 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506e58a-7621-43d8-bfaa-a0568d3bf739 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:06.907607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506e58a-7621-43d8-bfaa-a0568d3bf739 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:06.907607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506e58a-7621-43d8-bfaa-a0568d3bf739 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:06.909380 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 280/542] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:06.914772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad815b1-6cc6-4a0b-846f-7020e6f6cc3f None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:06.915061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad815b1-6cc6-4a0b-846f-7020e6f6cc3f None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:06.915337 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad815b1-6cc6-4a0b-846f-7020e6f6cc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:06.943924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-1ad815b1-6cc6-4a0b-846f-7020e6f6cc3f None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:06.945168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1ad815b1-6cc6-4a0b-846f-7020e6f6cc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:06.978300 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762cc26-08fe-4800-8353-94fd47aa95f8 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:06.978622 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762cc26-08fe-4800-8353-94fd47aa95f8 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:06.978872 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762cc26-08fe-4800-8353-94fd47aa95f8 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:06.980441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 281/543] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:06.986392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1390b7c-de2a-43a2-959b-ea8d46668bed None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:06.986660 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1390b7c-de2a-43a2-959b-ea8d46668bed None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:06.986956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1390b7c-de2a-43a2-959b-ea8d46668bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:06.993531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:07.014567 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-a1390b7c-de2a-43a2-959b-ea8d46668bed None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:07.015813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a1390b7c-de2a-43a2-959b-ea8d46668bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:07.061877 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 263/544] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:07.108193 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 282/545] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:45:07.390064 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad815b1-6cc6-4a0b-846f-7020e6f6cc3f None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:07.462854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:07.489705 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1390b7c-de2a-43a2-959b-ea8d46668bed None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:07.527733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad815b1-6cc6-4a0b-846f-7020e6f6cc3f None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:07.589639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8474488-dbe2-42aa-8b51-26e3abbd3447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:07.590225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8474488-dbe2-42aa-8b51-26e3abbd3447 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:07.590569 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8474488-dbe2-42aa-8b51-26e3abbd3447 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:07.590879 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8474488-dbe2-42aa-8b51-26e3abbd3447 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:07.616247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8474488-dbe2-42aa-8b51-26e3abbd3447 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:07.627589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-f8474488-dbe2-42aa-8b51-26e3abbd3447 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:07.627589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:07.627589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:07.627589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:07.627589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:07.627589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:07.627589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:07.627589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:07.627589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:07.627589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:07.627589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:07.627589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:07.627589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:07.627589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:07.627589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:07.627589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:07.628504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:07.628504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:07.628504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:07.628504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:07.628504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:07.628504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:07.628504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:07.628504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:07.628504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:07.628504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:07.628504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:07.628504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:07.628504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:07.628504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:07.628504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:07.628504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:07.629299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:07.629299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:07.629299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:07.629299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:07.629299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:07.629299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:07.629299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:07.629299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:07.629299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:07.629299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:07.629299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:07.629299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:07.629299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 264/546] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:07.652099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8474488-dbe2-42aa-8b51-26e3abbd3447 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:07.689526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1390b7c-de2a-43a2-959b-ea8d46668bed None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:07.726520 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:07.732543 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aca23b-bd0e-46cb-80e8-5c36cf984eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:07.733595 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aca23b-bd0e-46cb-80e8-5c36cf984eff None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:07.733880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aca23b-bd0e-46cb-80e8-5c36cf984eff None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:07.734155 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aca23b-bd0e-46cb-80e8-5c36cf984eff None admin] PATH_INFO: `/v3/roles/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:07.746011 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aca23b-bd0e-46cb-80e8-5c36cf984eff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:07.752477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-a4aca23b-bd0e-46cb-80e8-5c36cf984eff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:07.752477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:07.752477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:07.752477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:07.752477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:07.752477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:07.752477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:07.752477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:07.752477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:07.752477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:07.752477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:07.752477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:07.752477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:07.752477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:07.752477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:07.752477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:07.753371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:07.753371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:07.753371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:07.753371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:07.753371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:07.753371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:07.753371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:07.753371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:07.753371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:07.753371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:07.753371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:07.753371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:07.753371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:07.753371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:07.753371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:07.753371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:07.754154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:07.754154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:07.754154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:07.754154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:07.754154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:07.754154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:07.754154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:07.754154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:07.754154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:07.754154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:07.754154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:07.754154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:07.758833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 283/547] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:07.760077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aca23b-bd0e-46cb-80e8-5c36cf984eff None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:07.859932 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8474488-dbe2-42aa-8b51-26e3abbd3447 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:07.865673 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aca23b-bd0e-46cb-80e8-5c36cf984eff None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:07.896203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:07.905305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b52e1-daa5-43a9-acae-e3f1bcc32600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:07.906678 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b52e1-daa5-43a9-acae-e3f1bcc32600 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:07.907360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b52e1-daa5-43a9-acae-e3f1bcc32600 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:07.907437 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b52e1-daa5-43a9-acae-e3f1bcc32600 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:07.908269 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b52e1-daa5-43a9-acae-e3f1bcc32600 None admin] RBAC: Adding query filter params (name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:07.908581 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b52e1-daa5-43a9-acae-e3f1bcc32600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:07.910052 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b52e1-daa5-43a9-acae-e3f1bcc32600 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:07.919571 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 284/548] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:07.927214 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b52e1-daa5-43a9-acae-e3f1bcc32600 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:07.941466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b599aed-0164-40d3-81ba-6529d0604acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:07.942367 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b599aed-0164-40d3-81ba-6529d0604acf None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:07.942649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b599aed-0164-40d3-81ba-6529d0604acf None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:07.942941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b599aed-0164-40d3-81ba-6529d0604acf None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:07.943613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b599aed-0164-40d3-81ba-6529d0604acf None admin] RBAC: Adding query filter params (name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:07.943912 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b599aed-0164-40d3-81ba-6529d0604acf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:07.945289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b599aed-0164-40d3-81ba-6529d0604acf None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:07.960103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 265/549] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:07.966463 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b599aed-0164-40d3-81ba-6529d0604acf None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:08.040148 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:08.119826 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b599aed-0164-40d3-81ba-6529d0604acf None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:08.124167 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b52e1-daa5-43a9-acae-e3f1bcc32600 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:08.159732 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6014a02-7ee6-47dd-96c8-1664deb802ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:08.160410 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6014a02-7ee6-47dd-96c8-1664deb802ca None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.160690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6014a02-7ee6-47dd-96c8-1664deb802ca None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.160963 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6014a02-7ee6-47dd-96c8-1664deb802ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.173345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6014a02-7ee6-47dd-96c8-1664deb802ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:08.174824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6014a02-7ee6-47dd-96c8-1664deb802ca None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:08.192046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-b6014a02-7ee6-47dd-96c8-1664deb802ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:08.192046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:08.192046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:08.192046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:08.192046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:08.192046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:08.192046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:08.192046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:08.192046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:08.192046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:08.192046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:08.192046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:08.192046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:08.192046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:08.192046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:08.192046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:08.193183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:08.193183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:08.193183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:08.193183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:08.193183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:08.193183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:08.193183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:08.193183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:08.193183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:08.193183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:08.193183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:08.193183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:08.193183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:08.193183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:08.193183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:08.193183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:08.193183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:08.194088 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:08.194088 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:08.194088 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:08.198269 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 266/550] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:08.199639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6014a02-7ee6-47dd-96c8-1664deb802ca None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:08.229198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d183239-05ee-47e9-8f76-170262f6f2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:08.229846 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d183239-05ee-47e9-8f76-170262f6f2c4 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.230129 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d183239-05ee-47e9-8f76-170262f6f2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.230406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d183239-05ee-47e9-8f76-170262f6f2c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.243167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:08.254964 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d183239-05ee-47e9-8f76-170262f6f2c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:08.256446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d183239-05ee-47e9-8f76-170262f6f2c4 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:08.288045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-4d183239-05ee-47e9-8f76-170262f6f2c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:08.288045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:08.288045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:08.288045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:08.288045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:08.288045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:08.288045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:08.288045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:08.288045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:08.288045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:08.288045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:08.288045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:08.288045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:08.288045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:08.288045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:08.288045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:08.289044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:08.289044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:08.289044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:08.289044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:08.289044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:08.289044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:08.289044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:08.289044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:08.289044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:08.289044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:08.289044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:08.289044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:08.289044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:08.289044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:08.289044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:08.289044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:08.289044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:08.290084 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:08.290084 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:08.290084 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:08.290801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 285/551] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:08.296234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d183239-05ee-47e9-8f76-170262f6f2c4 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:08.397405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6014a02-7ee6-47dd-96c8-1664deb802ca None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:08.483517 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d183239-05ee-47e9-8f76-170262f6f2c4 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:08.483731 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e41e87-aa0b-4140-aba7-ab0db5727a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:08.484304 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e41e87-aa0b-4140-aba7-ab0db5727a1b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.484584 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e41e87-aa0b-4140-aba7-ab0db5727a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.484858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e41e87-aa0b-4140-aba7-ab0db5727a1b None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.486057 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e41e87-aa0b-4140-aba7-ab0db5727a1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:08.486359 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e41e87-aa0b-4140-aba7-ab0db5727a1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:08.487872 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e41e87-aa0b-4140-aba7-ab0db5727a1b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:08.493525 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:08.503559 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 267/552] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:08.505092 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cbb06-edc8-4e52-9949-e43e8b157b84 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.505562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cbb06-edc8-4e52-9949-e43e8b157b84 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.506002 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cbb06-edc8-4e52-9949-e43e8b157b84 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.511652 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 268/553] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:08.513171 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40fd1c-cd7c-4923-abad-163071a00583 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.513638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40fd1c-cd7c-4923-abad-163071a00583 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.515422 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40fd1c-cd7c-4923-abad-163071a00583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.528449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4dff76-be1a-4bf6-9c0c-529da4538e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:08.529156 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dff76-be1a-4bf6-9c0c-529da4538e14 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.529787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dff76-be1a-4bf6-9c0c-529da4538e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.529907 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4dff76-be1a-4bf6-9c0c-529da4538e14 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.530606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4dff76-be1a-4bf6-9c0c-529da4538e14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:08.531004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4dff76-be1a-4bf6-9c0c-529da4538e14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:08.533013 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4dff76-be1a-4bf6-9c0c-529da4538e14 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:08.537120 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-4a40fd1c-cd7c-4923-abad-163071a00583 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:08.538545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4a40fd1c-cd7c-4923-abad-163071a00583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:08.551923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 286/554] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:08.551923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4dff76-be1a-4bf6-9c0c-529da4538e14 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:08.635128 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 269/555] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:08.636427 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40fd1c-cd7c-4923-abad-163071a00583 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:08.664792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4dff76-be1a-4bf6-9c0c-529da4538e14 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:08.683149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:08.705251 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70750c30-68fe-4687-bc87-8a5bf069cd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:08.706016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70750c30-68fe-4687-bc87-8a5bf069cd03 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.706368 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70750c30-68fe-4687-bc87-8a5bf069cd03 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.706711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70750c30-68fe-4687-bc87-8a5bf069cd03 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.718714 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70750c30-68fe-4687-bc87-8a5bf069cd03 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:08.722526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70750c30-68fe-4687-bc87-8a5bf069cd03 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:08.740723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-70750c30-68fe-4687-bc87-8a5bf069cd03 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:08.740723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:08.740723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:08.740723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:08.740723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:08.740723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:08.740723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:08.740723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:08.740723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:08.740723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:08.740723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:08.740723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:08.740723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:08.740723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:08.740723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:08.740723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:08.741709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:08.742619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:08.742619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:08.742619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:08.742619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:08.742619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:08.742619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:08.742619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:08.742619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:08.742619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:08.742619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:08.745154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 287/556] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Sep 22 14:45:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:08.746601 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ac78f-7ca9-476b-86b3-20a9bf33f2d6 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.747183 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ac78f-7ca9-476b-86b3-20a9bf33f2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.747657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ac78f-7ca9-476b-86b3-20a9bf33f2d6 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.752931 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 288/557] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:08.752931 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ac78f-7ca9-476b-86b3-20a9bf33f2d6 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:08.785455 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:08.803368 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40fd1c-cd7c-4923-abad-163071a00583 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:08.853203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4642e90-5d14-4330-b04c-35d2fa0a4aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:08.853902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4642e90-5d14-4330-b04c-35d2fa0a4aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.854184 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4642e90-5d14-4330-b04c-35d2fa0a4aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.854452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4642e90-5d14-4330-b04c-35d2fa0a4aa9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.862004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4642e90-5d14-4330-b04c-35d2fa0a4aa9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:08.863524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4642e90-5d14-4330-b04c-35d2fa0a4aa9 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:08.873053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ac78f-7ca9-476b-86b3-20a9bf33f2d6 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:08.876053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-b4642e90-5d14-4330-b04c-35d2fa0a4aa9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:08.876053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:08.876053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:08.876053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:08.876053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:08.876053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:08.876053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:08.876053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:08.876053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:08.876053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:08.876053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:08.876053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:08.876053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:08.876053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:08.876053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:08.876053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:08.877936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:08.879167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:08.879167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:08.879167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:08.879167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:08.879167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:08.879167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:08.879167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:08.879167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:08.879167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:08.879167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:08.879167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 270/558] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:08.879167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a76192-cf19-470b-bb55-fb57fcb490ec None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.879857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a76192-cf19-470b-bb55-fb57fcb490ec None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.879857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a76192-cf19-470b-bb55-fb57fcb490ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.899362 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-61a76192-cf19-470b-bb55-fb57fcb490ec None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:08.900586 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-61a76192-cf19-470b-bb55-fb57fcb490ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:08.920703 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8f18b-1c97-4ad0-abac-05e222cb1ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:08.922095 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8f18b-1c97-4ad0-abac-05e222cb1ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:08.923189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8f18b-1c97-4ad0-abac-05e222cb1ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:08.923848 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8f18b-1c97-4ad0-abac-05e222cb1ca6 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:08.924617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8f18b-1c97-4ad0-abac-05e222cb1ca6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:08.924928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8f18b-1c97-4ad0-abac-05e222cb1ca6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:08.926361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8f18b-1c97-4ad0-abac-05e222cb1ca6 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:08.951114 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 289/559] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Sep 22 14:45:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:08.953669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8f18b-1c97-4ad0-abac-05e222cb1ca6 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:08.979886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 271/560] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:08.981336 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a76192-cf19-470b-bb55-fb57fcb490ec None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.019168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:09.048678 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:09.070293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8f18b-1c97-4ad0-abac-05e222cb1ca6 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.096136 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a76192-cf19-470b-bb55-fb57fcb490ec None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.105635 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb5361-74cf-45a2-8bd0-2da91c299519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.106200 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb5361-74cf-45a2-8bd0-2da91c299519 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.106473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb5361-74cf-45a2-8bd0-2da91c299519 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.106765 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb5361-74cf-45a2-8bd0-2da91c299519 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.107472 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb5361-74cf-45a2-8bd0-2da91c299519 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:09.107772 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb5361-74cf-45a2-8bd0-2da91c299519 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:09.109188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb5361-74cf-45a2-8bd0-2da91c299519 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:09.128741 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 290/561] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Sep 22 14:45:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:09.131843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb5361-74cf-45a2-8bd0-2da91c299519 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.132132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f21b60-852f-40ea-afc6-06914ea943fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.132847 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f21b60-852f-40ea-afc6-06914ea943fa None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.133133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f21b60-852f-40ea-afc6-06914ea943fa None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.133405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f21b60-852f-40ea-afc6-06914ea943fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.144157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f21b60-852f-40ea-afc6-06914ea943fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:09.145619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f21b60-852f-40ea-afc6-06914ea943fa None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:09.162246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-c7f21b60-852f-40ea-afc6-06914ea943fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:09.162246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:09.162246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:09.162246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:09.162246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:09.162246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:09.162246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:09.162246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:09.162246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:09.162246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:09.162246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:09.162246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:09.162246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:09.162246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:09.162246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:09.162246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:09.163320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:09.163320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:09.163320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:09.163320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:09.163320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:09.163320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:09.163320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:09.163320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:09.163320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:09.163320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:09.163320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:09.163320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:09.163320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:09.163320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:09.163320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:09.163320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:09.163320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:09.164210 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:09.164210 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:09.164210 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:09.164210 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 272/562] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:09.164616 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e5f4e-5b18-4eac-92cc-b2bf87648775 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.164899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e5f4e-5b18-4eac-92cc-b2bf87648775 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.165165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e5f4e-5b18-4eac-92cc-b2bf87648775 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.170457 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 273/563] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:09.170457 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e5f4e-5b18-4eac-92cc-b2bf87648775 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.237807 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:09.251985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb5361-74cf-45a2-8bd0-2da91c299519 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.280882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e5f4e-5b18-4eac-92cc-b2bf87648775 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.286335 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1350c0-d8a9-4fbc-8e9e-8b97eb21528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.286990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1350c0-d8a9-4fbc-8e9e-8b97eb21528f None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.287264 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1350c0-d8a9-4fbc-8e9e-8b97eb21528f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.287528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1350c0-d8a9-4fbc-8e9e-8b97eb21528f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.297455 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1350c0-d8a9-4fbc-8e9e-8b97eb21528f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:09.302906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-5b1350c0-d8a9-4fbc-8e9e-8b97eb21528f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:09.302906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:09.302906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:09.302906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:09.302906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:09.302906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:09.302906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:09.302906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:09.302906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:09.302906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:09.302906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:09.302906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:09.302906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:09.302906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:09.302906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:09.302906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:09.304033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:09.304033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:09.304033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:09.304033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:09.304033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:09.304033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:09.304033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:09.304033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:09.304033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:09.304033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:09.304033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:09.304033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:09.304033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:09.304033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:09.304033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:09.304033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:09.304878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:09.304878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:09.304878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:09.304878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:09.304878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:09.304878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:09.304878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:09.304878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:09.304878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:09.304878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:09.304878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:09.304878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:09.304878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 291/564] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:09.305606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1350c0-d8a9-4fbc-8e9e-8b97eb21528f None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.310128 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:09.319417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2790fbb-06df-478b-adfa-e8af66830b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.320034 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2790fbb-06df-478b-adfa-e8af66830b96 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.320311 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2790fbb-06df-478b-adfa-e8af66830b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.320584 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2790fbb-06df-478b-adfa-e8af66830b96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.392730 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2790fbb-06df-478b-adfa-e8af66830b96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:09.394199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2790fbb-06df-478b-adfa-e8af66830b96 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:09.411542 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1350c0-d8a9-4fbc-8e9e-8b97eb21528f None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.444166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734edf85-4094-4db3-bb37-d8189310e0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.444761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734edf85-4094-4db3-bb37-d8189310e0db None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.445037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734edf85-4094-4db3-bb37-d8189310e0db None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.445304 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734edf85-4094-4db3-bb37-d8189310e0db None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.445928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734edf85-4094-4db3-bb37-d8189310e0db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:09.446222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734edf85-4094-4db3-bb37-d8189310e0db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:09.447601 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734edf85-4094-4db3-bb37-d8189310e0db None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:09.455033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:09.460548 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 292/565] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:09.461757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924d729-adb3-425e-a99d-f4ac9b47ec55 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.462049 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924d729-adb3-425e-a99d-f4ac9b47ec55 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.462324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924d729-adb3-425e-a99d-f4ac9b47ec55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.480643 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-a924d729-adb3-425e-a99d-f4ac9b47ec55 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:09.481853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a924d729-adb3-425e-a99d-f4ac9b47ec55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:09.500602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 274/566] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:09.501398 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2790fbb-06df-478b-adfa-e8af66830b96 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.539647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:09.550120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 293/567] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:09.550795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924d729-adb3-425e-a99d-f4ac9b47ec55 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.620517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2790fbb-06df-478b-adfa-e8af66830b96 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.654601 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e107a3d-691f-4acb-8a35-87236519c225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.655133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924d729-adb3-425e-a99d-f4ac9b47ec55 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.655240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e107a3d-691f-4acb-8a35-87236519c225 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.655517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e107a3d-691f-4acb-8a35-87236519c225 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.655841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e107a3d-691f-4acb-8a35-87236519c225 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.668751 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e107a3d-691f-4acb-8a35-87236519c225 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:09.673560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-3e107a3d-691f-4acb-8a35-87236519c225 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:09.673560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:09.673560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:09.673560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:09.673560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:09.673560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:09.673560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:09.673560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:09.673560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:09.673560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:09.673560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:09.673560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:09.673560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:09.673560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:09.673560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:09.673560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:09.674483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:09.674483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:09.674483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:09.674483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:09.674483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:09.674483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:09.674483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:09.674483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:09.674483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:09.674483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:09.674483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:09.674483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:09.674483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:09.674483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:09.674483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:09.674483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:09.675724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:09.675724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:09.675724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:09.675724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:09.675724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:09.675724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:09.675724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:09.675724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:09.675724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:09.675724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:09.675724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:09.675724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:09.675724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 275/568] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:09.675724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e107a3d-691f-4acb-8a35-87236519c225 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.697805 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a9b1de-39b7-4b58-b486-0b2ee55c7a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.698414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9b1de-39b7-4b58-b486-0b2ee55c7a44 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.698826 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9b1de-39b7-4b58-b486-0b2ee55c7a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.699100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9b1de-39b7-4b58-b486-0b2ee55c7a44 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.699748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a9b1de-39b7-4b58-b486-0b2ee55c7a44 None admin] RBAC: Adding query filter params (name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:09.700051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a9b1de-39b7-4b58-b486-0b2ee55c7a44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:09.701705 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:09.702339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a9b1de-39b7-4b58-b486-0b2ee55c7a44 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:09.717718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 294/569] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:09.718601 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a9b1de-39b7-4b58-b486-0b2ee55c7a44 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.780591 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e107a3d-691f-4acb-8a35-87236519c225 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.782792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:09.816768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75171aad-8e29-425e-9755-6f209de88b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.817397 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75171aad-8e29-425e-9755-6f209de88b83 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.817673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75171aad-8e29-425e-9755-6f209de88b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.817943 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75171aad-8e29-425e-9755-6f209de88b83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.825326 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75171aad-8e29-425e-9755-6f209de88b83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:09.827044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75171aad-8e29-425e-9755-6f209de88b83 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:09.834520 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a9b1de-39b7-4b58-b486-0b2ee55c7a44 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.839036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-75171aad-8e29-425e-9755-6f209de88b83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:09.839036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:09.839036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:09.839036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:09.839036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:09.839036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:09.839036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:09.839036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:09.839036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:09.839036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:09.839036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:09.839036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:09.839036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:09.839036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:09.839036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:09.840098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:09.840098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:09.840098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:09.840098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:09.840098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:09.840098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:09.840098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:09.840098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:09.840098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:09.840098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:09.840098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:09.840098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:09.840098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:09.840098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:09.840098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:09.840098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:09.840098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:09.840988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:09.840988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:09.840988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:09.840988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:09.840988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:09.840988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:09.840988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:09.840988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:09.840988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 276/570] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:09.840988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75171aad-8e29-425e-9755-6f209de88b83 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.871656 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c060ca5-5d5a-456a-9161-82b3c9afc60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.872339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c060ca5-5d5a-456a-9161-82b3c9afc60d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.872626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c060ca5-5d5a-456a-9161-82b3c9afc60d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.872901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c060ca5-5d5a-456a-9161-82b3c9afc60d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.886475 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c060ca5-5d5a-456a-9161-82b3c9afc60d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:09.888001 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c060ca5-5d5a-456a-9161-82b3c9afc60d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:09.903075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-5c060ca5-5d5a-456a-9161-82b3c9afc60d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:09.903075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:09.903075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:09.903075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:09.903075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:09.903075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:09.903075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:09.903075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:09.903075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:09.903075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:09.903075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:09.903075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:09.903075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:09.903075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:09.903075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:09.903075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:09.904065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:09.904065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:09.904065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:09.904065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:09.904065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:09.904065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:09.904065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:09.904065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:09.904065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:09.904065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:09.904065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:09.904065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:09.904065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:09.904065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:09.904065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:09.904065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:09.904065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:09.904977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:09.904977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:09.904977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:09.904977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 295/571] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:09.904977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c060ca5-5d5a-456a-9161-82b3c9afc60d None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:09.929826 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:09.951025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75171aad-8e29-425e-9755-6f209de88b83 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:09.985223 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc231c-107c-46c1-9516-649efc38f4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:09.985898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc231c-107c-46c1-9516-649efc38f4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:09.986199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc231c-107c-46c1-9516-649efc38f4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:09.986498 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc231c-107c-46c1-9516-649efc38f4d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:09.987309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc231c-107c-46c1-9516-649efc38f4d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:09.987668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc231c-107c-46c1-9516-649efc38f4d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:09.989303 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc231c-107c-46c1-9516-649efc38f4d2 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:09.997834 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 277/572] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:09.998648 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc231c-107c-46c1-9516-649efc38f4d2 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.015972 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c060ca5-5d5a-456a-9161-82b3c9afc60d None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.024523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:10.055178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503df4dc-8f21-420b-8596-65cfd0501d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.055727 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503df4dc-8f21-420b-8596-65cfd0501d9e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.056010 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503df4dc-8f21-420b-8596-65cfd0501d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.056283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503df4dc-8f21-420b-8596-65cfd0501d9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.067560 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503df4dc-8f21-420b-8596-65cfd0501d9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.069014 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503df4dc-8f21-420b-8596-65cfd0501d9e None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.083081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-503df4dc-8f21-420b-8596-65cfd0501d9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:10.083081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:10.083081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:10.083081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:10.083081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:10.083081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:10.083081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:10.083081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:10.083081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:10.083081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:10.083081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:10.083081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:10.083081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:10.083081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:10.083081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:10.083081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:10.084067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:10.084067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:10.084067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:10.084067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:10.084067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:10.084067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:10.084067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:10.084067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:10.084067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:10.084067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:10.084067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:10.084067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:10.084067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:10.084067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:10.084067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:10.084067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:10.084067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:10.084979 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:10.084979 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:10.084979 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:10.084979 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 296/573] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:10.084979 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503df4dc-8f21-420b-8596-65cfd0501d9e None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.105439 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc231c-107c-46c1-9516-649efc38f4d2 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.140750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4eaa91-7fba-4817-bd89-9a8aa24cd205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.141401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4eaa91-7fba-4817-bd89-9a8aa24cd205 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.141676 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4eaa91-7fba-4817-bd89-9a8aa24cd205 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.141944 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4eaa91-7fba-4817-bd89-9a8aa24cd205 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.151782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4eaa91-7fba-4817-bd89-9a8aa24cd205 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:10.157790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-4a4eaa91-7fba-4817-bd89-9a8aa24cd205 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:10.157790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:10.157790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:10.157790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:10.157790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:10.157790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:10.157790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:10.157790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:10.157790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:10.157790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:10.157790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:10.157790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:10.157790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:10.157790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:10.157790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:10.157790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:10.158897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:10.158897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:10.158897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:10.158897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:10.158897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:10.158897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:10.158897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:10.158897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:10.158897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:10.158897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:10.158897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:10.158897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:10.158897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:10.158897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:10.158897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:10.158897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:10.159756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:10.159756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:10.159756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:10.159756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:10.159756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:10.159756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:10.159756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:10.159756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:10.159756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:10.159756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:10.159756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:10.159756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:10.159756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 278/574] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:10.159756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4eaa91-7fba-4817-bd89-9a8aa24cd205 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.163063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:10.187381 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503df4dc-8f21-420b-8596-65cfd0501d9e None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.220583 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6430b-36c4-4c12-93c4-8e47f52aaa53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.221191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6430b-36c4-4c12-93c4-8e47f52aaa53 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.221725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6430b-36c4-4c12-93c4-8e47f52aaa53 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.222003 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6430b-36c4-4c12-93c4-8e47f52aaa53 None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.222689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6430b-36c4-4c12-93c4-8e47f52aaa53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:10.223027 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6430b-36c4-4c12-93c4-8e47f52aaa53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.224476 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6430b-36c4-4c12-93c4-8e47f52aaa53 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.236828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 297/575] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:10.237430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6430b-36c4-4c12-93c4-8e47f52aaa53 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.256260 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:10.263353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4eaa91-7fba-4817-bd89-9a8aa24cd205 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.296144 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12adcb4-3633-48ad-bd64-4b3bf3284e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.296759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12adcb4-3633-48ad-bd64-4b3bf3284e7b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.297033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12adcb4-3633-48ad-bd64-4b3bf3284e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.297307 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12adcb4-3633-48ad-bd64-4b3bf3284e7b None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.298454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12adcb4-3633-48ad-bd64-4b3bf3284e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:10.298781 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12adcb4-3633-48ad-bd64-4b3bf3284e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.300147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12adcb4-3633-48ad-bd64-4b3bf3284e7b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.311921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 279/576] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:10.312689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12adcb4-3633-48ad-bd64-4b3bf3284e7b None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.343449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6430b-36c4-4c12-93c4-8e47f52aaa53 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.375439 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:10.377484 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefbfc2-38ae-4bba-a4c3-264579300894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.378158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefbfc2-38ae-4bba-a4c3-264579300894 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.378493 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefbfc2-38ae-4bba-a4c3-264579300894 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.378752 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefbfc2-38ae-4bba-a4c3-264579300894 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.392188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefbfc2-38ae-4bba-a4c3-264579300894 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.393663 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefbfc2-38ae-4bba-a4c3-264579300894 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.409759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-3eefbfc2-38ae-4bba-a4c3-264579300894 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:10.409759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:10.409759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:10.409759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:10.409759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:10.409759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:10.409759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:10.409759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:10.409759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:10.409759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:10.409759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:10.409759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:10.409759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:10.409759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:10.409759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:10.409759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:10.411024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:10.411024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:10.411024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:10.411024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:10.411024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:10.411024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:10.411024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:10.411024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:10.411024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:10.411024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:10.411024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:10.411024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:10.411024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:10.411024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:10.411024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:10.411024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:10.411024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:10.412137 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:10.412137 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:10.412137 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:10.412137 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 298/577] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:10.412137 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefbfc2-38ae-4bba-a4c3-264579300894 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.419327 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12adcb4-3633-48ad-bd64-4b3bf3284e7b None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.454987 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb185412-f8bf-4d6d-84f2-290281fa3ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.455598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb185412-f8bf-4d6d-84f2-290281fa3ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.456163 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb185412-f8bf-4d6d-84f2-290281fa3ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.456163 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb185412-f8bf-4d6d-84f2-290281fa3ab4 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.456755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb185412-f8bf-4d6d-84f2-290281fa3ab4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:10.457053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb185412-f8bf-4d6d-84f2-290281fa3ab4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.458428 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb185412-f8bf-4d6d-84f2-290281fa3ab4 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.469744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 280/578] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:10.470922 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb185412-f8bf-4d6d-84f2-290281fa3ab4 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.491759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:10.525161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefbfc2-38ae-4bba-a4c3-264579300894 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.558875 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cc865-69ef-440a-be05-dafe6680c693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.559481 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cc865-69ef-440a-be05-dafe6680c693 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.559763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cc865-69ef-440a-be05-dafe6680c693 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.560038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cc865-69ef-440a-be05-dafe6680c693 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.560701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cc865-69ef-440a-be05-dafe6680c693 None admin] RBAC: Adding query filter params (name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:10.561003 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cc865-69ef-440a-be05-dafe6680c693 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.562444 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1cc865-69ef-440a-be05-dafe6680c693 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.572955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 299/579] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:10.573792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cc865-69ef-440a-be05-dafe6680c693 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.580588 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb185412-f8bf-4d6d-84f2-290281fa3ab4 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.604423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:10.615174 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b3182-fde7-441e-bb79-2b78f334a1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.615833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b3182-fde7-441e-bb79-2b78f334a1e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.616107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b3182-fde7-441e-bb79-2b78f334a1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.616377 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b3182-fde7-441e-bb79-2b78f334a1e6 None admin] PATH_INFO: `/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/0c9be41c39494e54a14398a1f03c4d69/roles/35bdd5122aae45ebbe44158a433d2f99` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.631115 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b3182-fde7-441e-bb79-2b78f334a1e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=43bc5871e4bf441b874474bcae4e5385, user_id=0c9be41c39494e54a14398a1f03c4d69, role_id=35bdd5122aae45ebbe44158a433d2f99)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.632555 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b3182-fde7-441e-bb79-2b78f334a1e6 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.645734 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 281/580] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:10 2022] PUT /identity/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/0c9be41c39494e54a14398a1f03c4d69/roles/35bdd5122aae45ebbe44158a433d2f99 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:45:10.646304 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b3182-fde7-441e-bb79-2b78f334a1e6 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.683661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1cc865-69ef-440a-be05-dafe6680c693 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.725868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:10.733541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa13947-760f-464d-a6a5-a3d61c40df11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.734940 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa13947-760f-464d-a6a5-a3d61c40df11 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.734940 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa13947-760f-464d-a6a5-a3d61c40df11 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.734940 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa13947-760f-464d-a6a5-a3d61c40df11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.744151 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa13947-760f-464d-a6a5-a3d61c40df11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.746191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa13947-760f-464d-a6a5-a3d61c40df11 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.759988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-6aa13947-760f-464d-a6a5-a3d61c40df11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:10.759988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:10.759988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:10.759988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:10.759988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:10.759988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:10.759988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:10.759988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:10.759988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:10.759988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:10.759988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:10.759988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:10.759988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:10.759988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:10.759988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:10.761070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:10.761070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:10.761070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:10.761070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:10.761070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:10.761070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:10.761070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:10.761070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:10.761070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:10.761070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:10.761070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:10.761070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:10.761070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:10.761070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:10.761070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:10.761070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:10.761070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:10.768286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:10.768286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:10.768286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:10.768286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:10.768286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:10.768286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:10.768286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:10.768286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:10.768726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b3182-fde7-441e-bb79-2b78f334a1e6 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.768803 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 300/581] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:10.768803 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa13947-760f-464d-a6a5-a3d61c40df11 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.809943 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4387f-db09-403f-8cdf-181cf1a0ddff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.810990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4387f-db09-403f-8cdf-181cf1a0ddff None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.811561 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4387f-db09-403f-8cdf-181cf1a0ddff None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.811902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4387f-db09-403f-8cdf-181cf1a0ddff None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.812700 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4387f-db09-403f-8cdf-181cf1a0ddff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:10.813014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4387f-db09-403f-8cdf-181cf1a0ddff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.815139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4387f-db09-403f-8cdf-181cf1a0ddff None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.831507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 282/582] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:10.831817 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4387f-db09-403f-8cdf-181cf1a0ddff None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.870928 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:10.896742 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa13947-760f-464d-a6a5-a3d61c40df11 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.938573 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db42e83-1901-4fe8-94dd-c456346697c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.939240 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db42e83-1901-4fe8-94dd-c456346697c7 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.939523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db42e83-1901-4fe8-94dd-c456346697c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.939800 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db42e83-1901-4fe8-94dd-c456346697c7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:10.948846 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db42e83-1901-4fe8-94dd-c456346697c7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:10.949678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4387f-db09-403f-8cdf-181cf1a0ddff None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:10.950322 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db42e83-1901-4fe8-94dd-c456346697c7 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:10.963817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-3db42e83-1901-4fe8-94dd-c456346697c7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 14:45:10.963817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:10.963817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:10.963817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:10.963817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:10.963817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:10.963817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:10.963817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:10.963817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:10.963817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:10.963817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:10.963817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:10.963817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:10.963817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:10.963817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:10.963817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:10.964895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:10.965860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:10.965860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:10.965860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:10.965860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:10.965860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:10.965860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:10.965860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:10.965860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:10.965860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 14:45:10.965860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:10.965860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 301/583] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:10.965860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db42e83-1901-4fe8-94dd-c456346697c7 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:10.995425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24911ced-6d14-451a-93fe-d6285e00c5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:10.995997 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24911ced-6d14-451a-93fe-d6285e00c5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:10.996273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24911ced-6d14-451a-93fe-d6285e00c5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:10.996572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24911ced-6d14-451a-93fe-d6285e00c5c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.007622 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24911ced-6d14-451a-93fe-d6285e00c5c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.009073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24911ced-6d14-451a-93fe-d6285e00c5c8 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.024441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-24911ced-6d14-451a-93fe-d6285e00c5c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:11.024441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:11.024441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:11.024441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:11.024441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:11.024441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:11.024441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:11.024441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:11.024441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:11.024441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:11.024441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:11.024441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:11.024441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:11.024441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:11.024441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:11.024441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:11.025452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:11.025452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:11.025452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:11.025452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:11.025452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:11.025452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:11.025452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:11.025452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:11.025452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:11.025452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:11.025452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:11.025452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:11.025452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:11.025452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:11.025452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:11.025452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:11.025452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:11.026363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:11.026363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:11.026363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:11.026363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 283/584] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:11.027159 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24911ced-6d14-451a-93fe-d6285e00c5c8 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.034211 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:11.090555 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db42e83-1901-4fe8-94dd-c456346697c7 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.123306 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:11.129410 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbb893-9d6a-44a8-95db-0c48dfc740f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.130035 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbb893-9d6a-44a8-95db-0c48dfc740f0 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.130308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbb893-9d6a-44a8-95db-0c48dfc740f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.130579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbb893-9d6a-44a8-95db-0c48dfc740f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.131302 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbb893-9d6a-44a8-95db-0c48dfc740f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:11.131606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbb893-9d6a-44a8-95db-0c48dfc740f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.133058 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbb893-9d6a-44a8-95db-0c48dfc740f0 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.140101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24911ced-6d14-451a-93fe-d6285e00c5c8 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.147428 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 302/585] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:11.151394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbb893-9d6a-44a8-95db-0c48dfc740f0 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.175661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fba85f-7eb9-46ff-b78f-3558ce024958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.176301 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fba85f-7eb9-46ff-b78f-3558ce024958 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.176575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fba85f-7eb9-46ff-b78f-3558ce024958 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.176903 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fba85f-7eb9-46ff-b78f-3558ce024958 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.184622 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fba85f-7eb9-46ff-b78f-3558ce024958 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.186049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fba85f-7eb9-46ff-b78f-3558ce024958 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.197794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-d0fba85f-7eb9-46ff-b78f-3558ce024958 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 14:45:11.197794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:11.197794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:11.197794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:11.197794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:11.197794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:11.197794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:11.197794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:11.197794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:11.197794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:11.197794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:11.197794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:11.197794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:11.197794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:11.197794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:11.197794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:11.198837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:11.199800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:11.199800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:11.199800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:11.199800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:11.199800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:11.199800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:11.199800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:11.199800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:11.199800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 14:45:11.199800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:11.199800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 284/586] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:11.199800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fba85f-7eb9-46ff-b78f-3558ce024958 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.264255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbb893-9d6a-44a8-95db-0c48dfc740f0 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.287356 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:11.312802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fba85f-7eb9-46ff-b78f-3558ce024958 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.317904 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457cfe0b-f7f8-46f4-8571-daf24e2dca6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.318515 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cfe0b-f7f8-46f4-8571-daf24e2dca6f None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.318818 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cfe0b-f7f8-46f4-8571-daf24e2dca6f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.319088 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cfe0b-f7f8-46f4-8571-daf24e2dca6f None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.319811 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457cfe0b-f7f8-46f4-8571-daf24e2dca6f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:11.320110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457cfe0b-f7f8-46f4-8571-daf24e2dca6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.321975 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457cfe0b-f7f8-46f4-8571-daf24e2dca6f None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.342681 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 303/587] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:11.343432 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457cfe0b-f7f8-46f4-8571-daf24e2dca6f None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.355673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d119a6-5b81-4ca7-be3a-54a6d7df2522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.356316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d119a6-5b81-4ca7-be3a-54a6d7df2522 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.356593 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d119a6-5b81-4ca7-be3a-54a6d7df2522 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.356868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d119a6-5b81-4ca7-be3a-54a6d7df2522 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.357828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d119a6-5b81-4ca7-be3a-54a6d7df2522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:11.358133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d119a6-5b81-4ca7-be3a-54a6d7df2522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.359543 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d119a6-5b81-4ca7-be3a-54a6d7df2522 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.372170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 285/588] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:11.372811 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d119a6-5b81-4ca7-be3a-54a6d7df2522 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.385048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:11.472956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457cfe0b-f7f8-46f4-8571-daf24e2dca6f None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.489780 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d119a6-5b81-4ca7-be3a-54a6d7df2522 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.511997 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e26c3d-3896-4739-820e-95e1f349c522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.512732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e26c3d-3896-4739-820e-95e1f349c522 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.513251 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e26c3d-3896-4739-820e-95e1f349c522 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.513928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e26c3d-3896-4739-820e-95e1f349c522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.514805 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e26c3d-3896-4739-820e-95e1f349c522 None admin] RBAC: Adding query filter params (role.id=35bdd5122aae45ebbe44158a433d2f99, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=0a265623ae734936a340c97df852e227) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:11.515222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e26c3d-3896-4739-820e-95e1f349c522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.516784 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e26c3d-3896-4739-820e-95e1f349c522 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.523923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 304/589] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/role_assignments?user.id=0a265623ae734936a340c97df852e227&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=35bdd5122aae45ebbe44158a433d2f99 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:11.525145 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e26c3d-3896-4739-820e-95e1f349c522 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.538266 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc283a46-5192-4522-aa5b-548d5c2d8f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.539217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc283a46-5192-4522-aa5b-548d5c2d8f09 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.539675 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc283a46-5192-4522-aa5b-548d5c2d8f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.539951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc283a46-5192-4522-aa5b-548d5c2d8f09 None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.540716 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc283a46-5192-4522-aa5b-548d5c2d8f09 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:11.541022 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc283a46-5192-4522-aa5b-548d5c2d8f09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.542548 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc283a46-5192-4522-aa5b-548d5c2d8f09 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.565939 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 286/590] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:11.567000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc283a46-5192-4522-aa5b-548d5c2d8f09 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.590657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:11.646892 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:11.690910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc283a46-5192-4522-aa5b-548d5c2d8f09 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.696508 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e26c3d-3896-4739-820e-95e1f349c522 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.743864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e8cd9-ee2a-41b5-b1c7-fe579a2cdaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.745351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e8cd9-ee2a-41b5-b1c7-fe579a2cdaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.745781 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e8cd9-ee2a-41b5-b1c7-fe579a2cdaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.746212 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e8cd9-ee2a-41b5-b1c7-fe579a2cdaf0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.753349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0ff15-8a85-4402-b46a-4387f19b627a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.754281 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0ff15-8a85-4402-b46a-4387f19b627a None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.754765 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0ff15-8a85-4402-b46a-4387f19b627a None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.755150 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0ff15-8a85-4402-b46a-4387f19b627a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.755470 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e8cd9-ee2a-41b5-b1c7-fe579a2cdaf0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.757241 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e8cd9-ee2a-41b5-b1c7-fe579a2cdaf0 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.771851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0ff15-8a85-4402-b46a-4387f19b627a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.772496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-ad0e8cd9-ee2a-41b5-b1c7-fe579a2cdaf0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 14:45:11.772496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:11.772496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:11.772496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:11.772496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:11.772496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:11.772496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:11.772496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:11.772496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:11.772496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:11.772496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:11.772496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:11.772496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:11.772496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:11.772496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:11.772496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:11.773655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:11.774841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:11.774841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:11.774841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:11.774841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:11.774841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:11.774841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:11.774841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:11.774841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:11.774841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 14:45:11.774841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:11.776596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 287/591] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:11.776818 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0ff15-8a85-4402-b46a-4387f19b627a None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.777773 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e8cd9-ee2a-41b5-b1c7-fe579a2cdaf0 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.795699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-c5f0ff15-8a85-4402-b46a-4387f19b627a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:11.795699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:11.795699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:11.795699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:11.795699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:11.795699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:11.795699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:11.795699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:11.795699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:11.795699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:11.795699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:11.795699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:11.795699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:11.795699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:11.795699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:11.795699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:11.796913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:11.796913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:11.796913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:11.796913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:11.796913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:11.796913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:11.796913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:11.796913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:11.796913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:11.796913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:11.796913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:11.796913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:11.796913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:11.796913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:11.796913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:11.796913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:11.796913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:11.798066 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:11.798066 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:11.798066 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:11.804296 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 305/592] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:11.806191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0ff15-8a85-4402-b46a-4387f19b627a None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:11.892404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e8cd9-ee2a-41b5-b1c7-fe579a2cdaf0 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.899662 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:11.917880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0ff15-8a85-4402-b46a-4387f19b627a None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:11.927134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:11.935427 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799da2e5-9ea0-48ae-8815-1b2fe093e74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.936213 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799da2e5-9ea0-48ae-8815-1b2fe093e74b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.936592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799da2e5-9ea0-48ae-8815-1b2fe093e74b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.937034 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799da2e5-9ea0-48ae-8815-1b2fe093e74b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.955024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799da2e5-9ea0-48ae-8815-1b2fe093e74b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.957185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799da2e5-9ea0-48ae-8815-1b2fe093e74b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.964792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a6b4b-8907-437f-9cf8-2ab3c67a2ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:11.966220 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a6b4b-8907-437f-9cf8-2ab3c67a2ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:11.966711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a6b4b-8907-437f-9cf8-2ab3c67a2ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:11.967776 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a6b4b-8907-437f-9cf8-2ab3c67a2ea1 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:11.969011 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6a6b4b-8907-437f-9cf8-2ab3c67a2ea1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:11.970087 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6a6b4b-8907-437f-9cf8-2ab3c67a2ea1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:11.972172 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6a6b4b-8907-437f-9cf8-2ab3c67a2ea1 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:11.974665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-799da2e5-9ea0-48ae-8815-1b2fe093e74b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:11.974665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:11.974665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:11.974665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:11.974665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:11.974665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:11.974665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:11.974665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:11.974665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:11.974665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:11.974665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:11.974665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:11.974665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:11.974665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:11.974665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:11.974665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:11.975610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:11.975610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:11.975610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:11.975610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:11.975610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:11.975610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:11.975610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:11.975610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:11.975610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:11.975610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:11.975610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:11.975610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:11.975610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:11.975610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:11.975610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:11.975610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:11.975610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:11.976524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:11.976524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:11.976524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:11.976524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 288/593] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:11.977185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799da2e5-9ea0-48ae-8815-1b2fe093e74b None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.001292 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 306/594] 10.222.0.26 () {56 vars in 1139 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:12.003067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a6b4b-8907-437f-9cf8-2ab3c67a2ea1 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.089554 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799da2e5-9ea0-48ae-8815-1b2fe093e74b None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:12.135342 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3f69e-6a5f-4642-90ea-2d032e989a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:12.136198 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3f69e-6a5f-4642-90ea-2d032e989a7e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:12.136319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a6b4b-8907-437f-9cf8-2ab3c67a2ea1 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:12.137089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3f69e-6a5f-4642-90ea-2d032e989a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:12.137508 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3f69e-6a5f-4642-90ea-2d032e989a7e None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:12.138555 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3f69e-6a5f-4642-90ea-2d032e989a7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:12.139898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3f69e-6a5f-4642-90ea-2d032e989a7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:12.141544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a3f69e-6a5f-4642-90ea-2d032e989a7e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:12.159952 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 289/595] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:12.161208 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3f69e-6a5f-4642-90ea-2d032e989a7e None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.179565 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e5a74-0ad8-4ad6-b8e7-0505ccc6f965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:12.180365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e5a74-0ad8-4ad6-b8e7-0505ccc6f965 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:12.180747 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e5a74-0ad8-4ad6-b8e7-0505ccc6f965 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:12.181119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e5a74-0ad8-4ad6-b8e7-0505ccc6f965 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:12.181884 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e5a74-0ad8-4ad6-b8e7-0505ccc6f965 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:12.182297 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e5a74-0ad8-4ad6-b8e7-0505ccc6f965 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:12.183819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e5a74-0ad8-4ad6-b8e7-0505ccc6f965 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:12.196252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 307/596] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:12.197262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e5a74-0ad8-4ad6-b8e7-0505ccc6f965 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.202112 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:12.210508 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:12.306635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3f69e-6a5f-4642-90ea-2d032e989a7e None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:12.314082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e5a74-0ad8-4ad6-b8e7-0505ccc6f965 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:12.356541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161262a4-5d76-4e58-bc7a-c0b324d8115d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:12.357361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161262a4-5d76-4e58-bc7a-c0b324d8115d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:12.357744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e41c2-6293-464f-8eb5-a54b077eb93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:12.357940 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161262a4-5d76-4e58-bc7a-c0b324d8115d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:12.358314 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161262a4-5d76-4e58-bc7a-c0b324d8115d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:12.358884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e41c2-6293-464f-8eb5-a54b077eb93a None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:12.359458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e41c2-6293-464f-8eb5-a54b077eb93a None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:12.359876 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e41c2-6293-464f-8eb5-a54b077eb93a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:12.366098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161262a4-5d76-4e58-bc7a-c0b324d8115d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:12.367797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161262a4-5d76-4e58-bc7a-c0b324d8115d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:12.373186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e41c2-6293-464f-8eb5-a54b077eb93a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:12.374855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e41c2-6293-464f-8eb5-a54b077eb93a None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:12.380779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-161262a4-5d76-4e58-bc7a-c0b324d8115d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:12.380779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:12.380779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:12.380779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:12.380779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:12.380779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:12.380779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:12.380779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:12.380779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:12.380779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:12.380779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:12.380779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:12.380779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:12.380779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:12.380779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:12.381864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:12.381864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:12.381864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:12.381864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:12.381864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:12.381864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:12.381864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:12.381864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:12.381864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:12.381864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:12.381864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:12.381864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:12.381864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:12.381864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:12.381864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:12.381864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:12.381864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:12.382966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:12.382966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:12.382966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:12.382966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:12.382966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:12.382966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:12.382966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:12.382966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:12.384377 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 308/597] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:12.389910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161262a4-5d76-4e58-bc7a-c0b324d8115d None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.395784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-525e41c2-6293-464f-8eb5-a54b077eb93a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:12.395784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:12.395784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:12.395784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:12.395784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:12.395784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:12.395784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:12.395784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:12.395784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:12.395784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:12.395784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:12.395784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:12.395784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:12.395784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:12.395784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:12.395784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:12.396884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:12.396884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:12.396884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:12.396884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:12.396884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:12.396884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:12.396884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:12.396884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:12.396884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:12.396884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:12.396884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:12.396884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:12.396884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:12.396884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:12.396884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:12.396884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:12.396884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:12.397990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:12.397990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:12.397990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:12.399590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 290/598] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:12.400547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e41c2-6293-464f-8eb5-a54b077eb93a None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.482371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:12.512004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:12.527158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e41c2-6293-464f-8eb5-a54b077eb93a None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:12.540763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161262a4-5d76-4e58-bc7a-c0b324d8115d None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:12.563924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7d982-e80a-4324-8c94-c7d068470155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:12.564673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7d982-e80a-4324-8c94-c7d068470155 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:12.565120 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7d982-e80a-4324-8c94-c7d068470155 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:12.565433 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7d982-e80a-4324-8c94-c7d068470155 None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:12.566239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7d982-e80a-4324-8c94-c7d068470155 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:12.566650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7d982-e80a-4324-8c94-c7d068470155 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:12.568299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7d982-e80a-4324-8c94-c7d068470155 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:12.585834 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166f4b2-529a-41f4-976c-d58e049bb7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:12.586767 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166f4b2-529a-41f4-976c-d58e049bb7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:12.587191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166f4b2-529a-41f4-976c-d58e049bb7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:12.587613 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166f4b2-529a-41f4-976c-d58e049bb7a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:12.588590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 291/599] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:12.589954 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7d982-e80a-4324-8c94-c7d068470155 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.597740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166f4b2-529a-41f4-976c-d58e049bb7a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:12.599442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166f4b2-529a-41f4-976c-d58e049bb7a8 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:12.615757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-c166f4b2-529a-41f4-976c-d58e049bb7a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:12.615757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:12.615757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:12.615757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:12.615757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:12.615757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:12.615757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:12.615757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:12.615757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:12.615757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:12.615757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:12.615757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:12.615757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:12.615757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:12.615757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:12.616874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:12.616874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:12.616874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:12.616874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:12.616874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:12.616874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:12.616874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:12.616874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:12.616874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:12.616874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:12.616874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:12.616874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:12.616874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:12.616874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:12.616874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:12.616874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:12.616874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:12.617953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:12.617953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:12.617953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:12.617953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:12.617953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:12.617953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:12.617953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:12.617953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:12.620668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 309/600] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:12.621885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166f4b2-529a-41f4-976c-d58e049bb7a8 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.724500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7d982-e80a-4324-8c94-c7d068470155 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:12.750120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166f4b2-529a-41f4-976c-d58e049bb7a8 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:12.777079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0dd7e4-4aa2-47cd-9457-09ef6099d728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:12.777775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dd7e4-4aa2-47cd-9457-09ef6099d728 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:12.778008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dd7e4-4aa2-47cd-9457-09ef6099d728 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:12.778275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dd7e4-4aa2-47cd-9457-09ef6099d728 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:12.778953 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0dd7e4-4aa2-47cd-9457-09ef6099d728 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:12.779251 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0dd7e4-4aa2-47cd-9457-09ef6099d728 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:12.780618 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0dd7e4-4aa2-47cd-9457-09ef6099d728 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:12.782987 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:12.796718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6f3c7-159c-48e5-9df8-df02602864c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:12.798257 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6f3c7-159c-48e5-9df8-df02602864c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:12.798757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6f3c7-159c-48e5-9df8-df02602864c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:12.799231 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6f3c7-159c-48e5-9df8-df02602864c6 None admin] PATH_INFO: `/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/bb5248cb1c39412680cce0d9891fe1ff/roles/35bdd5122aae45ebbe44158a433d2f99` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:12.800982 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 292/601] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:12.801901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0dd7e4-4aa2-47cd-9457-09ef6099d728 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.815449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:12.816478 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce6f3c7-159c-48e5-9df8-df02602864c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=43bc5871e4bf441b874474bcae4e5385, user_id=bb5248cb1c39412680cce0d9891fe1ff, role_id=35bdd5122aae45ebbe44158a433d2f99)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:12.818446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce6f3c7-159c-48e5-9df8-df02602864c6 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:12.835506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 310/602] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:12 2022] PUT /identity/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/bb5248cb1c39412680cce0d9891fe1ff/roles/35bdd5122aae45ebbe44158a433d2f99 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:45:12.836792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6f3c7-159c-48e5-9df8-df02602864c6 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:12.939342 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0dd7e4-4aa2-47cd-9457-09ef6099d728 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:12.984745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6f3c7-159c-48e5-9df8-df02602864c6 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:12.992805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f962a9-df94-477e-b43f-117190e9c115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:12.993824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f962a9-df94-477e-b43f-117190e9c115 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:12.994350 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f962a9-df94-477e-b43f-117190e9c115 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:12.994873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f962a9-df94-477e-b43f-117190e9c115 None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:12.996509 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f962a9-df94-477e-b43f-117190e9c115 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:12.997042 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f962a9-df94-477e-b43f-117190e9c115 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:12.999697 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f962a9-df94-477e-b43f-117190e9c115 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:13.019059 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 293/603] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:13.020114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ba681-12b6-4a9c-8786-174b05f062f8 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:13.020402 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ba681-12b6-4a9c-8786-174b05f062f8 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:13.021019 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ba681-12b6-4a9c-8786-174b05f062f8 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:13.022003 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 294/604] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:13.028575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ba681-12b6-4a9c-8786-174b05f062f8 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:13.034341 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fb4c8-c6e3-4a7c-a0cb-2490e18ef131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:13.035095 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fb4c8-c6e3-4a7c-a0cb-2490e18ef131 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:13.035383 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fb4c8-c6e3-4a7c-a0cb-2490e18ef131 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:13.035654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fb4c8-c6e3-4a7c-a0cb-2490e18ef131 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:13.042975 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fb4c8-c6e3-4a7c-a0cb-2490e18ef131 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:13.044502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fb4c8-c6e3-4a7c-a0cb-2490e18ef131 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:13.057647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-bb7fb4c8-c6e3-4a7c-a0cb-2490e18ef131 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:13.057647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:13.057647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:13.057647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:13.057647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:13.057647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:13.057647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:13.057647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:13.057647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:13.057647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:13.057647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:13.057647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:13.057647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:13.057647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:13.057647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:13.058591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:13.058591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:13.058591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:13.058591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:13.058591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:13.058591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:13.058591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:13.058591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:13.058591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:13.058591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:13.058591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:13.058591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:13.058591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:13.058591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:13.058591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:13.058591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:13.058591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:13.059514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:13.059514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:13.059514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:13.059514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:13.059514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:13.059514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:13.059514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:13.059514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:13.059514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 311/605] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:13.060341 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5419-69cf-454a-9640-387ab5d5bb0c None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:13.060618 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5419-69cf-454a-9640-387ab5d5bb0c None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:13.060825 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5419-69cf-454a-9640-387ab5d5bb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:13.080372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-b42a5419-69cf-454a-9640-387ab5d5bb0c None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:13.081569 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b42a5419-69cf-454a-9640-387ab5d5bb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:13.118329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:13.146266 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:13.168574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 312/606] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:13.169572 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a5419-69cf-454a-9640-387ab5d5bb0c None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:13.171533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ba681-12b6-4a9c-8786-174b05f062f8 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:13.221861 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ffd43-3ded-40d1-88e1-5f27daa9d777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:13.222803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ffd43-3ded-40d1-88e1-5f27daa9d777 None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:13.223778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ffd43-3ded-40d1-88e1-5f27daa9d777 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:13.224220 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ffd43-3ded-40d1-88e1-5f27daa9d777 None admin] PATH_INFO: `/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/1d265d111b534944b7812cf067e317a2/roles/35bdd5122aae45ebbe44158a433d2f99` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:13.244606 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ffd43-3ded-40d1-88e1-5f27daa9d777 None admin] RBAC: Authorizing `identity:create_grant(project_id=43bc5871e4bf441b874474bcae4e5385, user_id=1d265d111b534944b7812cf067e317a2, role_id=35bdd5122aae45ebbe44158a433d2f99)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:13.246955 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ffd43-3ded-40d1-88e1-5f27daa9d777 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:13.280592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 295/607] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:13 2022] PUT /identity/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/1d265d111b534944b7812cf067e317a2/roles/35bdd5122aae45ebbe44158a433d2f99 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:45:13.304824 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a5419-69cf-454a-9640-387ab5d5bb0c None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:13.345801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a98bdf-89fb-4c55-931b-4bd005d0e712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:13.346833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a98bdf-89fb-4c55-931b-4bd005d0e712 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:13.347120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a98bdf-89fb-4c55-931b-4bd005d0e712 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:13.347395 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a98bdf-89fb-4c55-931b-4bd005d0e712 None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:13.348166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a98bdf-89fb-4c55-931b-4bd005d0e712 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:13.348526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a98bdf-89fb-4c55-931b-4bd005d0e712 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:13.350027 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a98bdf-89fb-4c55-931b-4bd005d0e712 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:13.365721 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 313/608] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:13.371828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ffd43-3ded-40d1-88e1-5f27daa9d777 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:13.527443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a98bdf-89fb-4c55-931b-4bd005d0e712 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:13.537718 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ffd43-3ded-40d1-88e1-5f27daa9d777 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:13.556925 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:13.565251 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:13.582788 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813edcca-329c-4058-9467-4510fbd72ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:13.583751 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813edcca-329c-4058-9467-4510fbd72ce1 None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:13.584200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813edcca-329c-4058-9467-4510fbd72ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:13.584735 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813edcca-329c-4058-9467-4510fbd72ce1 None admin] PATH_INFO: `/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/f77bbd8de87042aab3674db366635cf5/roles/35bdd5122aae45ebbe44158a433d2f99` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:13.603178 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813edcca-329c-4058-9467-4510fbd72ce1 None admin] RBAC: Authorizing `identity:create_grant(project_id=43bc5871e4bf441b874474bcae4e5385, user_id=f77bbd8de87042aab3674db366635cf5, role_id=35bdd5122aae45ebbe44158a433d2f99)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:13.604926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813edcca-329c-4058-9467-4510fbd72ce1 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:13.626944 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 296/609] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:13 2022] PUT /identity/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/f77bbd8de87042aab3674db366635cf5/roles/35bdd5122aae45ebbe44158a433d2f99 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:45:13.686221 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a98bdf-89fb-4c55-931b-4bd005d0e712 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:13.752721 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5ad08-8bff-4a35-b07a-6ea9e40bb27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:13.753679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5ad08-8bff-4a35-b07a-6ea9e40bb27e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:13.754132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5ad08-8bff-4a35-b07a-6ea9e40bb27e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:13.754572 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5ad08-8bff-4a35-b07a-6ea9e40bb27e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:13.775555 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5ad08-8bff-4a35-b07a-6ea9e40bb27e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:13.781498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-53f5ad08-8bff-4a35-b07a-6ea9e40bb27e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:13.781498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:13.781498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:13.781498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:13.781498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:13.781498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:13.781498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:13.781498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:13.781498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:13.781498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:13.781498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:13.781498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:13.781498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:13.781498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:13.781498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:13.781498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:13.782901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:13.782901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:13.782901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:13.782901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:13.782901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:13.782901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:13.782901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:13.782901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:13.782901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:13.782901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:13.782901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:13.782901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:13.782901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:13.782901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:13.782901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:13.782901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:13.783807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:13.783807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:13.783807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:13.783807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:13.783807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:13.783807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:13.783807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:13.783807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:13.783807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:13.783807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:13.783807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:13.783807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:13.791396 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 314/610] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:13.807903 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813edcca-329c-4058-9467-4510fbd72ce1 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:13.939049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813edcca-329c-4058-9467-4510fbd72ce1 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:13.977505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49115498-0dbb-4006-bb47-f681a4394beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:13.978473 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49115498-0dbb-4006-bb47-f681a4394beb None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:13.978980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49115498-0dbb-4006-bb47-f681a4394beb None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:13.979443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49115498-0dbb-4006-bb47-f681a4394beb None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:13.980326 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49115498-0dbb-4006-bb47-f681a4394beb None admin] RBAC: Adding query filter params (name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:13.980840 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49115498-0dbb-4006-bb47-f681a4394beb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:13.982403 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49115498-0dbb-4006-bb47-f681a4394beb None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:13.992201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 297/611] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:13.997572 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5ad08-8bff-4a35-b07a-6ea9e40bb27e None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:14.070866 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f423d-662c-4d63-99a5-e8a80f3193f5 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:14.071508 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f423d-662c-4d63-99a5-e8a80f3193f5 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:14.071980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f423d-662c-4d63-99a5-e8a80f3193f5 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:14.074593 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 298/612] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:14.080493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce65722-8da5-4d3e-9b7e-bb6c38ef8773 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:14.081049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce65722-8da5-4d3e-9b7e-bb6c38ef8773 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:14.081560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce65722-8da5-4d3e-9b7e-bb6c38ef8773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:14.092191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:14.111677 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5ad08-8bff-4a35-b07a-6ea9e40bb27e None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:14.119767 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-8ce65722-8da5-4d3e-9b7e-bb6c38ef8773 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:14.121253 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-8ce65722-8da5-4d3e-9b7e-bb6c38ef8773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:14.153833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d3cb5-5f9b-4402-bc4d-48b6b438640e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:14.154809 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d3cb5-5f9b-4402-bc4d-48b6b438640e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:14.155104 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d3cb5-5f9b-4402-bc4d-48b6b438640e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:14.155378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d3cb5-5f9b-4402-bc4d-48b6b438640e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:14.174335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:14.183009 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d3cb5-5f9b-4402-bc4d-48b6b438640e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:14.184491 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d3cb5-5f9b-4402-bc4d-48b6b438640e None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:14.217853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-be3d3cb5-5f9b-4402-bc4d-48b6b438640e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:14.217853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:14.217853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:14.217853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:14.217853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:14.217853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:14.217853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:14.217853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:14.217853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:14.217853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:14.217853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:14.217853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:14.217853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:14.217853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:14.217853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:14.217853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:14.218889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:14.218889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:14.218889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:14.218889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:14.218889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:14.218889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:14.218889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:14.218889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:14.218889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:14.218889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:14.218889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:14.218889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:14.218889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:14.218889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:14.218889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:14.218889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:14.218889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:14.220447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:14.220447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:14.220447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:14.220447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 315/613] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:14.226125 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d3cb5-5f9b-4402-bc4d-48b6b438640e None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:14.273901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 299/614] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:14.367236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d3cb5-5f9b-4402-bc4d-48b6b438640e None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:14.408768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31001d-dac9-414f-9136-3164ebd40d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:14.409390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31001d-dac9-414f-9136-3164ebd40d7e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:14.409670 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31001d-dac9-414f-9136-3164ebd40d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:14.409942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31001d-dac9-414f-9136-3164ebd40d7e None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:14.410605 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31001d-dac9-414f-9136-3164ebd40d7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:14.410943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31001d-dac9-414f-9136-3164ebd40d7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:14.412382 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31001d-dac9-414f-9136-3164ebd40d7e None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:14.415297 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:14.426658 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 316/615] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:14.431901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce65722-8da5-4d3e-9b7e-bb6c38ef8773 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:14.579858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce65722-8da5-4d3e-9b7e-bb6c38ef8773 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:14.627640 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec05196-72b4-45b9-9970-ada733dbe9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:14.628282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec05196-72b4-45b9-9970-ada733dbe9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:14.628564 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec05196-72b4-45b9-9970-ada733dbe9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:14.628835 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec05196-72b4-45b9-9970-ada733dbe9d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:14.640485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec05196-72b4-45b9-9970-ada733dbe9d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:14.642466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec05196-72b4-45b9-9970-ada733dbe9d4 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:14.654636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-5ec05196-72b4-45b9-9970-ada733dbe9d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:14.654636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:14.654636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:14.654636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:14.654636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:14.654636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:14.654636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:14.654636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:14.654636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:14.654636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:14.654636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:14.654636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:14.654636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:14.654636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:14.654636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:14.654636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:14.655646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:14.656578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:14.656578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:14.656578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:14.656578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:14.656578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:14.656578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:14.656578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:14.656578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:14.656578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:14.656578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:14.656578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 300/616] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Sep 22 14:45:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:14.662292 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31001d-dac9-414f-9136-3164ebd40d7e None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:14.671850 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec05196-72b4-45b9-9970-ada733dbe9d4 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:14.695563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:14.788586 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec05196-72b4-45b9-9970-ada733dbe9d4 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:14.816171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31001d-dac9-414f-9136-3164ebd40d7e None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:14.842569 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec27d65-5ea6-4953-b916-d0ba005ae394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:14.843742 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec27d65-5ea6-4953-b916-d0ba005ae394 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:14.845029 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec27d65-5ea6-4953-b916-d0ba005ae394 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:14.845637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec27d65-5ea6-4953-b916-d0ba005ae394 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:14.866225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec27d65-5ea6-4953-b916-d0ba005ae394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:14.872112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-0ec27d65-5ea6-4953-b916-d0ba005ae394 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:14.872112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:14.872112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:14.872112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:14.872112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:14.872112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:14.872112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:14.872112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:14.872112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:14.872112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:14.872112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:14.872112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:14.872112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:14.872112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:14.872112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:14.872112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:14.873107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:14.873107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:14.873107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:14.873107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:14.873107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:14.873107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:14.873107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:14.873107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:14.873107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:14.873107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:14.873107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:14.873107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:14.873107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:14.873107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:14.873107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:14.873107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:14.873935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:14.873935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:14.873935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:14.873935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:14.873935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:14.873935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:14.873935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:14.873935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:14.873935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:14.873935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:14.873935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:14.873935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:14.881460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 301/617] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:14.888628 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec27d65-5ea6-4953-b916-d0ba005ae394 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:14.903458 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b537cd-bd37-4056-b21f-c746815fb989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:14.904593 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b537cd-bd37-4056-b21f-c746815fb989 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:14.905506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b537cd-bd37-4056-b21f-c746815fb989 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:14.906232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b537cd-bd37-4056-b21f-c746815fb989 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:14.907233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b537cd-bd37-4056-b21f-c746815fb989 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:14.907713 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b537cd-bd37-4056-b21f-c746815fb989 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:14.909315 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b537cd-bd37-4056-b21f-c746815fb989 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:14.936397 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 317/618] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Sep 22 14:45:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:14.956144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b537cd-bd37-4056-b21f-c746815fb989 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:14.999121 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:15.021242 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec27d65-5ea6-4953-b916-d0ba005ae394 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:15.070515 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:15.090170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8569ee-99f8-4415-8323-33942f61fba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:15.090798 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8569ee-99f8-4415-8323-33942f61fba9 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.091075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8569ee-99f8-4415-8323-33942f61fba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.091338 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8569ee-99f8-4415-8323-33942f61fba9 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.091996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8569ee-99f8-4415-8323-33942f61fba9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:15.092292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8569ee-99f8-4415-8323-33942f61fba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:15.095011 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8569ee-99f8-4415-8323-33942f61fba9 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:15.112782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 302/619] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:15.119481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8569ee-99f8-4415-8323-33942f61fba9 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:15.159241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b537cd-bd37-4056-b21f-c746815fb989 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:15.199283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a8961-d26d-45a7-9197-9faed27d7ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:15.200876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a8961-d26d-45a7-9197-9faed27d7ddf None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.201151 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a8961-d26d-45a7-9197-9faed27d7ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.201465 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a8961-d26d-45a7-9197-9faed27d7ddf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.318826 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8569ee-99f8-4415-8323-33942f61fba9 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:15.351995 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a8961-d26d-45a7-9197-9faed27d7ddf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:15.353406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a8961-d26d-45a7-9197-9faed27d7ddf None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:15.358768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa4dbe-dbd0-4402-bfe2-965001933383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:15.359416 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa4dbe-dbd0-4402-bfe2-965001933383 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.359826 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa4dbe-dbd0-4402-bfe2-965001933383 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.360175 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa4dbe-dbd0-4402-bfe2-965001933383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.372068 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa4dbe-dbd0-4402-bfe2-965001933383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:15.373641 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa4dbe-dbd0-4402-bfe2-965001933383 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:15.375661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:15.390829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-3cfa4dbe-dbd0-4402-bfe2-965001933383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:15.390829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:15.390829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:15.390829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:15.390829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:15.390829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:15.390829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:15.390829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:15.390829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:15.390829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:15.390829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:15.390829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:15.390829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:15.390829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:15.390829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:15.390829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:15.392562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:15.392562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:15.392562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:15.392562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:15.392562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:15.392562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:15.392562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:15.392562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:15.392562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:15.392562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:15.392562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:15.392562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:15.392562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:15.392562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:15.392562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:15.392562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:15.392562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:15.393640 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:15.393640 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:15.393640 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:15.395742 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 303/620] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:15.407682 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa4dbe-dbd0-4402-bfe2-965001933383 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:15.474016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:15.518465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa4dbe-dbd0-4402-bfe2-965001933383 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:15.525353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 318/621] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:15.527019 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07081875-fb37-4ad0-a74d-bd1c4bc1627b None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.527617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07081875-fb37-4ad0-a74d-bd1c4bc1627b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.528107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07081875-fb37-4ad0-a74d-bd1c4bc1627b None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.534517 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 319/622] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:15.535635 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07081875-fb37-4ad0-a74d-bd1c4bc1627b None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:15.559217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa5439b-331c-4b86-84e2-2e45fb175915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:15.559826 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5439b-331c-4b86-84e2-2e45fb175915 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.560099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5439b-331c-4b86-84e2-2e45fb175915 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.560366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa5439b-331c-4b86-84e2-2e45fb175915 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.560990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5439b-331c-4b86-84e2-2e45fb175915 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:15.561333 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5439b-331c-4b86-84e2-2e45fb175915 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:15.562652 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa5439b-331c-4b86-84e2-2e45fb175915 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:15.575284 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 304/623] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:15.576480 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abdcf3-30ed-4bdb-abe8-4d3f820cf180 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.576764 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abdcf3-30ed-4bdb-abe8-4d3f820cf180 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.577034 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abdcf3-30ed-4bdb-abe8-4d3f820cf180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.596763 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-e5abdcf3-30ed-4bdb-abe8-4d3f820cf180 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:15.597885 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e5abdcf3-30ed-4bdb-abe8-4d3f820cf180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:15.647806 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07081875-fb37-4ad0-a74d-bd1c4bc1627b None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:15.648690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:15.674757 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 305/624] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:15.674757 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abdcf3-30ed-4bdb-abe8-4d3f820cf180 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:15.710246 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da6b63-63ba-42d8-b07e-545d595599cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:15.711283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da6b63-63ba-42d8-b07e-545d595599cb None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.711854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da6b63-63ba-42d8-b07e-545d595599cb None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.712347 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da6b63-63ba-42d8-b07e-545d595599cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.731935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da6b63-63ba-42d8-b07e-545d595599cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:15.733780 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da6b63-63ba-42d8-b07e-545d595599cb None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:15.757779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-19da6b63-63ba-42d8-b07e-545d595599cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:15.757779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:15.757779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:15.757779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:15.757779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:15.757779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:15.757779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:15.757779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:15.757779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:15.757779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:15.757779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:15.757779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:15.757779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:15.757779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:15.757779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:15.757779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:15.758813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:15.758813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:15.758813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:15.758813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:15.758813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:15.758813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:15.758813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:15.758813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:15.758813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:15.758813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:15.758813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:15.758813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:15.758813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:15.758813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:15.758813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:15.758813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:15.758813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:15.759718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:15.759718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:15.759718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:15.766865 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 320/625] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:15.769031 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da6b63-63ba-42d8-b07e-545d595599cb None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:15.776323 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abdcf3-30ed-4bdb-abe8-4d3f820cf180 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:15.802873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:15.817692 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e80cb-9928-4c46-8e15-898bf87a2e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:15.817692 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e80cb-9928-4c46-8e15-898bf87a2e3b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.817692 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e80cb-9928-4c46-8e15-898bf87a2e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.817692 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e80cb-9928-4c46-8e15-898bf87a2e3b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.823551 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e80cb-9928-4c46-8e15-898bf87a2e3b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:15.825011 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e80cb-9928-4c46-8e15-898bf87a2e3b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:15.837605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-2d3e80cb-9928-4c46-8e15-898bf87a2e3b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:15.837605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:15.837605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:15.837605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:15.837605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:15.837605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:15.837605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:15.837605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:15.837605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:15.837605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:15.837605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:15.837605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:15.837605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:15.837605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:15.837605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:15.837605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:15.838769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:15.839959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:15.839959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:15.839959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:15.839959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:15.839959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:15.839959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:15.839959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:15.839959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:15.839959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:15.839959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:15.850402 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 306/626] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:15.851674 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e80cb-9928-4c46-8e15-898bf87a2e3b None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:15.915648 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:15.945109 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da6b63-63ba-42d8-b07e-545d595599cb None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:15.982096 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfbff53-1715-4dd7-afb3-9ab2e4a756e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:15.991007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbff53-1715-4dd7-afb3-9ab2e4a756e6 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:15.991321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbff53-1715-4dd7-afb3-9ab2e4a756e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:15.991598 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbff53-1715-4dd7-afb3-9ab2e4a756e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:15.992260 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfbff53-1715-4dd7-afb3-9ab2e4a756e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:15.992561 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfbff53-1715-4dd7-afb3-9ab2e4a756e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:15.993948 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfbff53-1715-4dd7-afb3-9ab2e4a756e6 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:16.011505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e80cb-9928-4c46-8e15-898bf87a2e3b None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.012555 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 321/627] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:16.013828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd40456-2875-49a7-80eb-4a656ee6f3b3 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.014110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd40456-2875-49a7-80eb-4a656ee6f3b3 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.014381 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd40456-2875-49a7-80eb-4a656ee6f3b3 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.020492 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 322/628] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:16.021906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72510fa7-ec69-4cd1-b5f5-cdfb19645428 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.022285 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72510fa7-ec69-4cd1-b5f5-cdfb19645428 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.022651 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72510fa7-ec69-4cd1-b5f5-cdfb19645428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.041878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-72510fa7-ec69-4cd1-b5f5-cdfb19645428 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:16.043422 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-72510fa7-ec69-4cd1-b5f5-cdfb19645428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:16.066198 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9c551-daf3-48c0-9d17-7ece3813e12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.067667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9c551-daf3-48c0-9d17-7ece3813e12b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.068114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9c551-daf3-48c0-9d17-7ece3813e12b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.068554 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9c551-daf3-48c0-9d17-7ece3813e12b None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.069767 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9c551-daf3-48c0-9d17-7ece3813e12b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:16.070478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9c551-daf3-48c0-9d17-7ece3813e12b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:16.072904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9c551-daf3-48c0-9d17-7ece3813e12b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:16.098303 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 307/629] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Sep 22 14:45:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:16.099769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9c551-daf3-48c0-9d17-7ece3813e12b None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.119878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:16.136505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 323/630] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:16.137885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72510fa7-ec69-4cd1-b5f5-cdfb19645428 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.250360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72510fa7-ec69-4cd1-b5f5-cdfb19645428 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.256430 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9c551-daf3-48c0-9d17-7ece3813e12b None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.258239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:16.298602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2fd26-6bb0-49fb-b2c9-fd5d61b22740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.300477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2fd26-6bb0-49fb-b2c9-fd5d61b22740 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.301007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2fd26-6bb0-49fb-b2c9-fd5d61b22740 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.301563 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2fd26-6bb0-49fb-b2c9-fd5d61b22740 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.304638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8b99f-0b65-4bba-bb26-97c2da574a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.305730 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8b99f-0b65-4bba-bb26-97c2da574a09 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.306006 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8b99f-0b65-4bba-bb26-97c2da574a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.306273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8b99f-0b65-4bba-bb26-97c2da574a09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.314480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2fd26-6bb0-49fb-b2c9-fd5d61b22740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:16.315756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8b99f-0b65-4bba-bb26-97c2da574a09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:16.315996 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2fd26-6bb0-49fb-b2c9-fd5d61b22740 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:16.317188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8b99f-0b65-4bba-bb26-97c2da574a09 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:16.330839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-b8d8b99f-0b65-4bba-bb26-97c2da574a09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:16.330839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:16.330839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:16.330839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:16.330839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:16.330839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:16.330839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:16.330839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:16.330839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:16.330839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:16.330839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:16.330839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:16.330839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:16.330839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:16.330839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:16.331750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:16.331750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:16.331750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:16.331750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:16.331750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:16.331750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:16.331750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:16.331750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:16.331750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:16.331750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:16.331750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:16.331750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:16.331750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:16.331750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:16.331750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:16.331750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:16.331750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:16.332634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:16.332634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:16.332634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:16.332634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:16.332634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:16.332634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:16.332634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:16.332634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:16.333550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-88d2fd26-6bb0-49fb-b2c9-fd5d61b22740 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:16.333550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:16.333550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:16.333550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:16.333550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:16.333550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:16.333550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:16.333550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:16.333550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:16.333550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:16.333550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:16.333550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:16.333550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:16.333550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:16.333550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:16.333550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:16.334641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:16.334641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:16.334641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:16.334641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:16.334641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:16.334641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:16.334641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:16.334641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:16.334641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:16.334641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:16.334641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:16.334641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:16.334641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:16.334641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:16.334641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:16.334641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:16.334641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:16.335755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:16.335755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:16.335755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:16.335755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 324/631] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:16.335755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2fd26-6bb0-49fb-b2c9-fd5d61b22740 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.336988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 308/632] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:16.338088 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938ac0f-0ae6-448b-94f4-843527bd3ccc None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.338448 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938ac0f-0ae6-448b-94f4-843527bd3ccc None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.339498 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5938ac0f-0ae6-448b-94f4-843527bd3ccc None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.340018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 309/633] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:16.340706 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938ac0f-0ae6-448b-94f4-843527bd3ccc None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.396621 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:16.454654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2fd26-6bb0-49fb-b2c9-fd5d61b22740 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.458533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5938ac0f-0ae6-448b-94f4-843527bd3ccc None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.489555 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ef0f9-91b3-460b-8a02-1e031a385eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.490202 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ef0f9-91b3-460b-8a02-1e031a385eda None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.490560 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ef0f9-91b3-460b-8a02-1e031a385eda None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.490871 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ef0f9-91b3-460b-8a02-1e031a385eda None admin] PATH_INFO: `/v3/roles/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.491824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d777e-00a3-4811-8e85-a88721da7f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.492424 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d777e-00a3-4811-8e85-a88721da7f53 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.492701 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d777e-00a3-4811-8e85-a88721da7f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.492971 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d777e-00a3-4811-8e85-a88721da7f53 None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.493660 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d777e-00a3-4811-8e85-a88721da7f53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:16.493960 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d777e-00a3-4811-8e85-a88721da7f53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:16.495471 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d777e-00a3-4811-8e85-a88721da7f53 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:16.502465 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99ef0f9-91b3-460b-8a02-1e031a385eda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:16.507718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-c99ef0f9-91b3-460b-8a02-1e031a385eda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:16.507718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:16.507718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:16.507718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:16.507718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:16.507718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:16.507718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:16.507718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:16.507718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:16.507718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:16.507718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:16.507718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:16.507718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:16.507718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:16.507718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:16.507718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:16.508659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:16.508659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:16.508659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:16.508659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:16.508659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:16.508659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:16.508659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:16.508659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:16.508659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:16.508659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:16.508659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:16.508659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:16.508659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:16.508659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:16.508659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:16.508659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:16.510177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:16.510177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:16.510177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:16.510177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:16.510177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:16.510177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:16.510177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:16.510177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:16.510177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:16.510177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:16.510177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:16.510177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:16.511165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 310/634] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:16.511238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 325/635] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:16.511319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d777e-00a3-4811-8e85-a88721da7f53 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.511397 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5315b-a85f-4f68-81ef-0b25adbbe084 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.511397 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5315b-a85f-4f68-81ef-0b25adbbe084 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.511397 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5315b-a85f-4f68-81ef-0b25adbbe084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.528503 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-aac5315b-a85f-4f68-81ef-0b25adbbe084 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:16.529709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-aac5315b-a85f-4f68-81ef-0b25adbbe084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:16.538892 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:16.596324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 326/636] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:16.597435 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5315b-a85f-4f68-81ef-0b25adbbe084 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.627898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d777e-00a3-4811-8e85-a88721da7f53 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.640598 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:16.663531 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28192f-704b-4d43-bc17-fcd697537c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.664140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28192f-704b-4d43-bc17-fcd697537c92 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.664420 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28192f-704b-4d43-bc17-fcd697537c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.664766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28192f-704b-4d43-bc17-fcd697537c92 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.665321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28192f-704b-4d43-bc17-fcd697537c92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:16.665623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28192f-704b-4d43-bc17-fcd697537c92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:16.667016 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28192f-704b-4d43-bc17-fcd697537c92 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:16.679656 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 311/637] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:16.680471 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28192f-704b-4d43-bc17-fcd697537c92 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.711754 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5315b-a85f-4f68-81ef-0b25adbbe084 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.747935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1671c390-faac-45f9-ace2-88b381b7ad67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.748553 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671c390-faac-45f9-ace2-88b381b7ad67 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.748882 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671c390-faac-45f9-ace2-88b381b7ad67 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.749161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671c390-faac-45f9-ace2-88b381b7ad67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.761728 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1671c390-faac-45f9-ace2-88b381b7ad67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:16.763881 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:16.767421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-1671c390-faac-45f9-ace2-88b381b7ad67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:16.767421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:16.767421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:16.767421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:16.767421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:16.767421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:16.767421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:16.767421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:16.767421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:16.767421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:16.767421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:16.767421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:16.767421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:16.767421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:16.767421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:16.767421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:16.768373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:16.768373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:16.768373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:16.768373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:16.768373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:16.768373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:16.768373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:16.768373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:16.768373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:16.768373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:16.768373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:16.768373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:16.768373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:16.768373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:16.768373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:16.768373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:16.769216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:16.769216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:16.769216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:16.769216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:16.769216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:16.769216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:16.769216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:16.769216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:16.769216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:16.769216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:16.769216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:16.769216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:16.769216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 327/638] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:16.769943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1671c390-faac-45f9-ace2-88b381b7ad67 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.790268 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28192f-704b-4d43-bc17-fcd697537c92 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.837917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98454223-8b7a-4c68-b680-dfbdc15cf157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.838530 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98454223-8b7a-4c68-b680-dfbdc15cf157 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.838855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98454223-8b7a-4c68-b680-dfbdc15cf157 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.839079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98454223-8b7a-4c68-b680-dfbdc15cf157 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.839813 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98454223-8b7a-4c68-b680-dfbdc15cf157 None admin] RBAC: Adding query filter params (role.id=35bdd5122aae45ebbe44158a433d2f99, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=0c9be41c39494e54a14398a1f03c4d69) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:16.840110 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98454223-8b7a-4c68-b680-dfbdc15cf157 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:16.841524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98454223-8b7a-4c68-b680-dfbdc15cf157 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:16.847883 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 312/639] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:16 2022] GET /identity/v3/role_assignments?user.id=0c9be41c39494e54a14398a1f03c4d69&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=35bdd5122aae45ebbe44158a433d2f99 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:16.848662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98454223-8b7a-4c68-b680-dfbdc15cf157 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.896865 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:16.902160 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1671c390-faac-45f9-ace2-88b381b7ad67 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:16.950888 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958454a-2e2d-4a44-b389-4bd36844e632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:16.951526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958454a-2e2d-4a44-b389-4bd36844e632 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:16.951803 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958454a-2e2d-4a44-b389-4bd36844e632 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:16.952076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958454a-2e2d-4a44-b389-4bd36844e632 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:16.952800 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958454a-2e2d-4a44-b389-4bd36844e632 None admin] RBAC: Adding query filter params (name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:16.953154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958454a-2e2d-4a44-b389-4bd36844e632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:16.954613 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958454a-2e2d-4a44-b389-4bd36844e632 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:16.964586 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 328/640] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:16.965456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958454a-2e2d-4a44-b389-4bd36844e632 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:16.982431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98454223-8b7a-4c68-b680-dfbdc15cf157 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.021533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d8a3f-fe52-4e36-b5bd-9d13b0eae3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.022017 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d8a3f-fe52-4e36-b5bd-9d13b0eae3bd None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.022283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d8a3f-fe52-4e36-b5bd-9d13b0eae3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.022558 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d8a3f-fe52-4e36-b5bd-9d13b0eae3bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.029827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d8a3f-fe52-4e36-b5bd-9d13b0eae3bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.031318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d8a3f-fe52-4e36-b5bd-9d13b0eae3bd None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.044060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-6f0d8a3f-fe52-4e36-b5bd-9d13b0eae3bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:17.044060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:17.044060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:17.044060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:17.044060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:17.044060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:17.044060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:17.044060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:17.044060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:17.044060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:17.044060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:17.044060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:17.044060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:17.044060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:17.044060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:17.044969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:17.044969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:17.044969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:17.044969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:17.044969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:17.044969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:17.044969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:17.044969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:17.044969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:17.044969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:17.044969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:17.044969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:17.044969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:17.044969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:17.044969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:17.044969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:17.044969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:17.045870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:17.045870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:17.045870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:17.045870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:17.045870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:17.045870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:17.045870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:17.045870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:17.045870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 313/641] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:17.046347 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d8a3f-fe52-4e36-b5bd-9d13b0eae3bd None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.059732 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:17.091753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958454a-2e2d-4a44-b389-4bd36844e632 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.134470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa28b9-13fe-4320-9042-91576d83f548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.135166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa28b9-13fe-4320-9042-91576d83f548 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.135450 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa28b9-13fe-4320-9042-91576d83f548 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.135717 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa28b9-13fe-4320-9042-91576d83f548 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.136424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa28b9-13fe-4320-9042-91576d83f548 None admin] RBAC: Adding query filter params (name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:17.136722 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa28b9-13fe-4320-9042-91576d83f548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.138504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa28b9-13fe-4320-9042-91576d83f548 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.147850 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 329/642] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:17.148691 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa28b9-13fe-4320-9042-91576d83f548 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.174037 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d8a3f-fe52-4e36-b5bd-9d13b0eae3bd None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.182172 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:17.220180 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94518f2a-6013-486d-8016-235d850c7cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.220838 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94518f2a-6013-486d-8016-235d850c7cfc None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.221113 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94518f2a-6013-486d-8016-235d850c7cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.221380 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94518f2a-6013-486d-8016-235d850c7cfc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.233821 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94518f2a-6013-486d-8016-235d850c7cfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:17.239766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-94518f2a-6013-486d-8016-235d850c7cfc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:17.239766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:17.239766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:17.239766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:17.239766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:17.239766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:17.239766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:17.239766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:17.239766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:17.239766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:17.239766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:17.239766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:17.239766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:17.239766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:17.239766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:17.239766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:17.240802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:17.240802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:17.240802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:17.240802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:17.240802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:17.240802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:17.240802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:17.240802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:17.240802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:17.240802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:17.240802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:17.240802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:17.240802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:17.240802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:17.240802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:17.240802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:17.241662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:17.241662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:17.241662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:17.241662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:17.241662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:17.241662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:17.241662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:17.241662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:17.241662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:17.241662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:17.241662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:17.241662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:17.241662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 314/643] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:17.241662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94518f2a-6013-486d-8016-235d850c7cfc None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.275243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa28b9-13fe-4320-9042-91576d83f548 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.310876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d92bd9-5140-4576-912e-9f8354602fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.311792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d92bd9-5140-4576-912e-9f8354602fbe None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.312076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d92bd9-5140-4576-912e-9f8354602fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.312357 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d92bd9-5140-4576-912e-9f8354602fbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.323281 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d92bd9-5140-4576-912e-9f8354602fbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.324779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d92bd9-5140-4576-912e-9f8354602fbe None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.326000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:17.340294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-33d92bd9-5140-4576-912e-9f8354602fbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:17.340294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:17.340294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:17.340294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:17.340294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:17.340294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:17.340294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:17.340294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:17.340294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:17.340294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:17.340294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:17.340294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:17.340294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:17.340294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:17.340294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:17.340294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:17.341477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:17.341477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:17.341477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:17.341477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:17.341477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:17.341477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:17.341477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:17.341477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:17.341477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:17.341477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:17.341477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:17.341477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:17.341477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:17.341477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:17.341477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:17.341477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:17.341477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:17.342795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:17.342795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:17.342795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:17.346886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 330/644] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:17.347752 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d92bd9-5140-4576-912e-9f8354602fbe None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.371981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94518f2a-6013-486d-8016-235d850c7cfc None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.415429 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944865e2-4824-44b9-b6fd-a7740f6ad727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.416153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944865e2-4824-44b9-b6fd-a7740f6ad727 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.416403 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944865e2-4824-44b9-b6fd-a7740f6ad727 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.416659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944865e2-4824-44b9-b6fd-a7740f6ad727 None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.417346 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944865e2-4824-44b9-b6fd-a7740f6ad727 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:17.417650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944865e2-4824-44b9-b6fd-a7740f6ad727 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.419925 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944865e2-4824-44b9-b6fd-a7740f6ad727 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.424698 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:17.438468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 315/645] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:17.438468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944865e2-4824-44b9-b6fd-a7740f6ad727 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.459288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d92bd9-5140-4576-912e-9f8354602fbe None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.498414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6bf73-2d40-4396-94ce-0dda655505b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.499214 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6bf73-2d40-4396-94ce-0dda655505b5 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.499420 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6bf73-2d40-4396-94ce-0dda655505b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.499777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6bf73-2d40-4396-94ce-0dda655505b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.511604 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b6bf73-2d40-4396-94ce-0dda655505b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.513056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b6bf73-2d40-4396-94ce-0dda655505b5 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.528851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-a9b6bf73-2d40-4396-94ce-0dda655505b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:17.528851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:17.528851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:17.528851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:17.528851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:17.528851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:17.528851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:17.528851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:17.528851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:17.528851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:17.528851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:17.528851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:17.528851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:17.528851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:17.528851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:17.528851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:17.529812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:17.529812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:17.529812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:17.529812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:17.529812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:17.529812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:17.529812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:17.529812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:17.529812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:17.529812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:17.529812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:17.529812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:17.529812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:17.529812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:17.529812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:17.529812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:17.529812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:17.530717 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:17.530717 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:17.530717 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:17.530717 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 331/646] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:17.530717 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6bf73-2d40-4396-94ce-0dda655505b5 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.553425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944865e2-4824-44b9-b6fd-a7740f6ad727 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.581050 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:17.591125 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6030645-42e2-4532-b2d2-96f85e3f7d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.591732 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6030645-42e2-4532-b2d2-96f85e3f7d68 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.592014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6030645-42e2-4532-b2d2-96f85e3f7d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.592285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6030645-42e2-4532-b2d2-96f85e3f7d68 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.592975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6030645-42e2-4532-b2d2-96f85e3f7d68 None admin] RBAC: Adding query filter params (name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:17.593352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6030645-42e2-4532-b2d2-96f85e3f7d68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.595512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6030645-42e2-4532-b2d2-96f85e3f7d68 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.609186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 316/647] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:17.609186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6030645-42e2-4532-b2d2-96f85e3f7d68 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.642088 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6bf73-2d40-4396-94ce-0dda655505b5 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.668617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:17.678681 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5034cdb-40cb-45c5-9f2b-b7c37d1a0ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.679236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5034cdb-40cb-45c5-9f2b-b7c37d1a0ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.679509 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5034cdb-40cb-45c5-9f2b-b7c37d1a0ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.679774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5034cdb-40cb-45c5-9f2b-b7c37d1a0ce6 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.680403 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5034cdb-40cb-45c5-9f2b-b7c37d1a0ce6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:17.680702 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5034cdb-40cb-45c5-9f2b-b7c37d1a0ce6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.682056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5034cdb-40cb-45c5-9f2b-b7c37d1a0ce6 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.699226 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 332/648] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:17.699226 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5034cdb-40cb-45c5-9f2b-b7c37d1a0ce6 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.730082 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6030645-42e2-4532-b2d2-96f85e3f7d68 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.773185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb00370-e4f6-406c-8534-8e59f272e646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.773842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb00370-e4f6-406c-8534-8e59f272e646 None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.774112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb00370-e4f6-406c-8534-8e59f272e646 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.774378 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb00370-e4f6-406c-8534-8e59f272e646 None admin] PATH_INFO: `/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/0a265623ae734936a340c97df852e227/roles/35bdd5122aae45ebbe44158a433d2f99` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.791341 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb00370-e4f6-406c-8534-8e59f272e646 None admin] RBAC: Authorizing `identity:create_grant(project_id=43bc5871e4bf441b874474bcae4e5385, user_id=0a265623ae734936a340c97df852e227, role_id=35bdd5122aae45ebbe44158a433d2f99)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.792709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb00370-e4f6-406c-8534-8e59f272e646 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.796950 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5034cdb-40cb-45c5-9f2b-b7c37d1a0ce6 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:17.818858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 317/649] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:17 2022] PUT /identity/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/0a265623ae734936a340c97df852e227/roles/35bdd5122aae45ebbe44158a433d2f99 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:45:17.819927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb00370-e4f6-406c-8534-8e59f272e646 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.840738 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3b1de-a4ba-41b0-8b40-140410475138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:17.841360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3b1de-a4ba-41b0-8b40-140410475138 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:17.841636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3b1de-a4ba-41b0-8b40-140410475138 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:17.842052 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3b1de-a4ba-41b0-8b40-140410475138 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:17.842712 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3b1de-a4ba-41b0-8b40-140410475138 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:17.843084 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3b1de-a4ba-41b0-8b40-140410475138 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:17.844599 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3b1de-a4ba-41b0-8b40-140410475138 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:17.863202 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 333/650] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:17.863202 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3b1de-a4ba-41b0-8b40-140410475138 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:17.890599 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:17.922195 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:17.979784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb00370-e4f6-406c-8534-8e59f272e646 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.018425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3b1de-a4ba-41b0-8b40-140410475138 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.020904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abb3a9-e3f1-496e-84eb-6dfe850256e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.021660 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abb3a9-e3f1-496e-84eb-6dfe850256e1 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.021936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abb3a9-e3f1-496e-84eb-6dfe850256e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.022200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abb3a9-e3f1-496e-84eb-6dfe850256e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.038320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abb3a9-e3f1-496e-84eb-6dfe850256e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.040149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abb3a9-e3f1-496e-84eb-6dfe850256e1 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.057481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-c2abb3a9-e3f1-496e-84eb-6dfe850256e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:18.057481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:18.057481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:18.057481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:18.057481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:18.057481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:18.057481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:18.057481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:18.057481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:18.057481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:18.057481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:18.057481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:18.057481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:18.057481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:18.057481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:18.057481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:18.058461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:18.058461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:18.058461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:18.058461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:18.058461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:18.058461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:18.058461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:18.058461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:18.058461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:18.058461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:18.058461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:18.058461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:18.058461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:18.058461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:18.058461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:18.058461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:18.058461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:18.059390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:18.059390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:18.059390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:18.059569 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64caf88-7dd0-4c31-a558-7516cf1b67f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.059569 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64caf88-7dd0-4c31-a558-7516cf1b67f0 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.059569 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64caf88-7dd0-4c31-a558-7516cf1b67f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.059569 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64caf88-7dd0-4c31-a558-7516cf1b67f0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.061636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 318/651] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:18.062638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abb3a9-e3f1-496e-84eb-6dfe850256e1 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.073384 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64caf88-7dd0-4c31-a558-7516cf1b67f0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.075155 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64caf88-7dd0-4c31-a558-7516cf1b67f0 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.087699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-c64caf88-7dd0-4c31-a558-7516cf1b67f0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 14:45:18.087699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:18.087699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:18.087699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:18.087699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:18.087699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:18.087699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:18.087699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:18.087699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:18.087699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:18.087699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:18.087699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:18.087699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:18.087699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:18.087699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:18.087699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:18.088903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:18.090054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:18.090054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:18.090054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:18.090054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:18.090054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:18.090054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:18.090054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:18.090054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:18.090054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 14:45:18.090054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:18.091856 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 334/652] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:18.092917 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64caf88-7dd0-4c31-a558-7516cf1b67f0 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.165571 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:18.177842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abb3a9-e3f1-496e-84eb-6dfe850256e1 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.216963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64caf88-7dd0-4c31-a558-7516cf1b67f0 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.225140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1eaa2-881c-41ca-a1e0-f522d541bf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.225785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1eaa2-881c-41ca-a1e0-f522d541bf6d None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.226070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1eaa2-881c-41ca-a1e0-f522d541bf6d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.226344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b1eaa2-881c-41ca-a1e0-f522d541bf6d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.227489 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:18.239690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1eaa2-881c-41ca-a1e0-f522d541bf6d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.241863 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b1eaa2-881c-41ca-a1e0-f522d541bf6d None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.258768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-85b1eaa2-881c-41ca-a1e0-f522d541bf6d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 14:45:18.258768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:18.258768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:18.258768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:18.258768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:18.258768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:18.258768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:18.258768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:18.258768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:18.258768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:18.258768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:18.258768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:18.258768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:18.258768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:18.258768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:18.258768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:18.259771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:18.260744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:18.260744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:18.260744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:18.260744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:18.260744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:18.260744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:18.260744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:18.260744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:18.260744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 14:45:18.260744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:18.263185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592973d-1a9b-4b47-99e2-a72e507dd5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.264195 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592973d-1a9b-4b47-99e2-a72e507dd5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.264562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 319/653] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:18.264864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592973d-1a9b-4b47-99e2-a72e507dd5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.265323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592973d-1a9b-4b47-99e2-a72e507dd5a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.265427 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1eaa2-881c-41ca-a1e0-f522d541bf6d None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.266641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592973d-1a9b-4b47-99e2-a72e507dd5a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:18.267192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592973d-1a9b-4b47-99e2-a72e507dd5a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.268746 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592973d-1a9b-4b47-99e2-a72e507dd5a9 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.282806 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 335/654] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:18.284403 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592973d-1a9b-4b47-99e2-a72e507dd5a9 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.397509 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592973d-1a9b-4b47-99e2-a72e507dd5a9 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.402293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b1eaa2-881c-41ca-a1e0-f522d541bf6d None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.441642 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab6a3c-39b2-4e5a-a812-7ba220802ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.442617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab6a3c-39b2-4e5a-a812-7ba220802ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.443130 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab6a3c-39b2-4e5a-a812-7ba220802ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.443627 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab6a3c-39b2-4e5a-a812-7ba220802ff7 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.444559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab6a3c-39b2-4e5a-a812-7ba220802ff7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:18.445077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab6a3c-39b2-4e5a-a812-7ba220802ff7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.446673 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab6a3c-39b2-4e5a-a812-7ba220802ff7 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.449622 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de679c9b-55d0-45ce-a7e2-350f7938e61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.450407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de679c9b-55d0-45ce-a7e2-350f7938e61b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.450847 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de679c9b-55d0-45ce-a7e2-350f7938e61b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.451254 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de679c9b-55d0-45ce-a7e2-350f7938e61b None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.452157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de679c9b-55d0-45ce-a7e2-350f7938e61b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:18.452595 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de679c9b-55d0-45ce-a7e2-350f7938e61b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.454949 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de679c9b-55d0-45ce-a7e2-350f7938e61b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.466120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 336/655] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:18.467104 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab6a3c-39b2-4e5a-a812-7ba220802ff7 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.475531 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 320/656] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:18.476711 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de679c9b-55d0-45ce-a7e2-350f7938e61b None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.489774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:18.495526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:18.590346 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab6a3c-39b2-4e5a-a812-7ba220802ff7 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.597010 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de679c9b-55d0-45ce-a7e2-350f7938e61b None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.632069 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7d741-743e-46a7-8bb2-2932b11426e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.632726 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7d741-743e-46a7-8bb2-2932b11426e4 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.633008 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7d741-743e-46a7-8bb2-2932b11426e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.633285 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7d741-743e-46a7-8bb2-2932b11426e4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.640967 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7d741-743e-46a7-8bb2-2932b11426e4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.643118 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7d741-743e-46a7-8bb2-2932b11426e4 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.649033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275dcd6-deb4-411a-ae47-046f8d30f460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.650284 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275dcd6-deb4-411a-ae47-046f8d30f460 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.650781 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275dcd6-deb4-411a-ae47-046f8d30f460 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.651244 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275dcd6-deb4-411a-ae47-046f8d30f460 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.655798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-43b7d741-743e-46a7-8bb2-2932b11426e4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 14:45:18.655798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:18.655798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:18.655798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:18.655798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:18.655798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:18.655798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:18.655798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:18.655798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:18.655798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:18.655798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:18.655798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:18.655798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:18.655798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:18.655798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:18.655798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:18.657332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:18.658756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:18.658756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:18.658756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:18.658756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:18.658756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:18.658756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:18.658756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:18.658756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:18.658756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 14:45:18.658756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:18.661951 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 337/657] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:18.662905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7d741-743e-46a7-8bb2-2932b11426e4 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.734019 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275dcd6-deb4-411a-ae47-046f8d30f460 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.736092 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275dcd6-deb4-411a-ae47-046f8d30f460 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.746724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:18.779903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:18.804149 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7d741-743e-46a7-8bb2-2932b11426e4 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.847916 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 321/658] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:18.848788 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275dcd6-deb4-411a-ae47-046f8d30f460 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:18.853116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905d6fb-8e7b-43a5-a4a0-470e8a399bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:18.853955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905d6fb-8e7b-43a5-a4a0-470e8a399bbd None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:18.854425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905d6fb-8e7b-43a5-a4a0-470e8a399bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:18.854911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905d6fb-8e7b-43a5-a4a0-470e8a399bbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:18.940356 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905d6fb-8e7b-43a5-a4a0-470e8a399bbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:18.941967 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905d6fb-8e7b-43a5-a4a0-470e8a399bbd None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:18.961833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275dcd6-deb4-411a-ae47-046f8d30f460 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:18.972695 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:19.001060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142deb7-6605-441f-8b7c-1e0a8db9e202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:19.002085 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142deb7-6605-441f-8b7c-1e0a8db9e202 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.002537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142deb7-6605-441f-8b7c-1e0a8db9e202 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.003092 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142deb7-6605-441f-8b7c-1e0a8db9e202 None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.004009 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5142deb7-6605-441f-8b7c-1e0a8db9e202 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:19.004507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5142deb7-6605-441f-8b7c-1e0a8db9e202 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:19.006344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5142deb7-6605-441f-8b7c-1e0a8db9e202 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:19.026081 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 322/659] 10.222.0.26 () {56 vars in 1139 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:19.027384 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142deb7-6605-441f-8b7c-1e0a8db9e202 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:19.037517 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:19.058298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 338/660] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:19.059212 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905d6fb-8e7b-43a5-a4a0-470e8a399bbd None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:19.144214 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142deb7-6605-441f-8b7c-1e0a8db9e202 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:19.175411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905d6fb-8e7b-43a5-a4a0-470e8a399bbd None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:19.182331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e330404-15f9-4702-bdf0-b4dec8961da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:19.183412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e330404-15f9-4702-bdf0-b4dec8961da2 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.183885 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e330404-15f9-4702-bdf0-b4dec8961da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.184363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e330404-15f9-4702-bdf0-b4dec8961da2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.196623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e330404-15f9-4702-bdf0-b4dec8961da2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:19.198210 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e330404-15f9-4702-bdf0-b4dec8961da2 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:19.213718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a6d79-fa76-4236-8406-304d6eff6006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:19.214252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-6e330404-15f9-4702-bdf0-b4dec8961da2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:19.214252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:19.214252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:19.214252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:19.214252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:19.214252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:19.214252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:19.214252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:19.214252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:19.214252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:19.214252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:19.214252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:19.214252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:19.214252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:19.214252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:19.214252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:19.215217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:19.215217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:19.215217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:19.215217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:19.215217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:19.215217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:19.215217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:19.215217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:19.215217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:19.215217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:19.215217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:19.215217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:19.215217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:19.215217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:19.215217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:19.215217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:19.215217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:19.216732 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:19.216732 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:19.216732 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:19.216901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a6d79-fa76-4236-8406-304d6eff6006 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.216901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a6d79-fa76-4236-8406-304d6eff6006 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.216901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a6d79-fa76-4236-8406-304d6eff6006 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.217087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 323/661] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:19.217087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e330404-15f9-4702-bdf0-b4dec8961da2 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:19.240239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:19.277120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:19.295660 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a6d79-fa76-4236-8406-304d6eff6006 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:19.297957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a6d79-fa76-4236-8406-304d6eff6006 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:19.334662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e330404-15f9-4702-bdf0-b4dec8961da2 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:19.372067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4e6c9-d04c-4155-8928-ac839c21f564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:19.372707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4e6c9-d04c-4155-8928-ac839c21f564 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.372985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4e6c9-d04c-4155-8928-ac839c21f564 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.373260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4e6c9-d04c-4155-8928-ac839c21f564 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.384809 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4e6c9-d04c-4155-8928-ac839c21f564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:19.386275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4e6c9-d04c-4155-8928-ac839c21f564 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:19.401234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-2cd4e6c9-d04c-4155-8928-ac839c21f564 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:19.401234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:19.401234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:19.401234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:19.401234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:19.401234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:19.401234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:19.401234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:19.401234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:19.401234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:19.401234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:19.401234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:19.401234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:19.401234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:19.401234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:19.401234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:19.402344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:19.402344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:19.402344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:19.402344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:19.402344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:19.402344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:19.402344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:19.402344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:19.402344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:19.402344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:19.402344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:19.402344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:19.402344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:19.402344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:19.402344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:19.402344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:19.402344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:19.403673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:19.403673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:19.403673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:19.406973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 324/662] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:19.408552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0025e28-6736-4b41-a02f-b6188bae62f3 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.408842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0025e28-6736-4b41-a02f-b6188bae62f3 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.409117 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0025e28-6736-4b41-a02f-b6188bae62f3 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.410523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 325/663] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:19.416117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 339/664] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:19.416117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a6d79-fa76-4236-8406-304d6eff6006 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:19.416338 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0025e28-6736-4b41-a02f-b6188bae62f3 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:19.485844 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:19.507675 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:19.521774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0025e28-6736-4b41-a02f-b6188bae62f3 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:19.528985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a6d79-fa76-4236-8406-304d6eff6006 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:19.558071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269ad57-f601-4b89-a1bd-3bf8193e2393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:19.558705 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269ad57-f601-4b89-a1bd-3bf8193e2393 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.558992 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269ad57-f601-4b89-a1bd-3bf8193e2393 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.559256 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269ad57-f601-4b89-a1bd-3bf8193e2393 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.559882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269ad57-f601-4b89-a1bd-3bf8193e2393 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:19.560178 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269ad57-f601-4b89-a1bd-3bf8193e2393 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:19.561524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269ad57-f601-4b89-a1bd-3bf8193e2393 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:19.571165 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87a651-2dcf-4118-ab4c-1c1cc8cb23f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:19.571939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87a651-2dcf-4118-ab4c-1c1cc8cb23f8 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.572237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87a651-2dcf-4118-ab4c-1c1cc8cb23f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.572501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87a651-2dcf-4118-ab4c-1c1cc8cb23f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.573146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87a651-2dcf-4118-ab4c-1c1cc8cb23f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:19.578273 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87a651-2dcf-4118-ab4c-1c1cc8cb23f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:19.578467 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 326/665] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:19.578467 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f927bda-bcf1-4ee6-94f0-0d3dfbfc3a71 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.578615 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87a651-2dcf-4118-ab4c-1c1cc8cb23f8 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:19.578685 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f927bda-bcf1-4ee6-94f0-0d3dfbfc3a71 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.578685 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f927bda-bcf1-4ee6-94f0-0d3dfbfc3a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.593174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 340/666] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:19.593174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87a651-2dcf-4118-ab4c-1c1cc8cb23f8 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:19.598785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-1f927bda-bcf1-4ee6-94f0-0d3dfbfc3a71 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:19.599991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1f927bda-bcf1-4ee6-94f0-0d3dfbfc3a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:19.669340 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 327/667] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:19.670116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f927bda-bcf1-4ee6-94f0-0d3dfbfc3a71 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:19.703611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87a651-2dcf-4118-ab4c-1c1cc8cb23f8 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:19.742011 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a45bf9-f733-4673-9bc8-c3c7ad3385aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:19.742640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a45bf9-f733-4673-9bc8-c3c7ad3385aa None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.742948 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a45bf9-f733-4673-9bc8-c3c7ad3385aa None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.743217 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a45bf9-f733-4673-9bc8-c3c7ad3385aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.753147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:19.758594 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a45bf9-f733-4673-9bc8-c3c7ad3385aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:19.760237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a45bf9-f733-4673-9bc8-c3c7ad3385aa None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:19.763593 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:19.778004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-59a45bf9-f733-4673-9bc8-c3c7ad3385aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:19.778004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:19.778004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:19.778004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:19.778004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:19.778004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:19.778004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:19.778004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:19.778004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:19.778004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:19.778004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:19.778004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:19.778004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:19.778004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:19.778004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:19.778004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:19.779144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:19.779144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:19.779144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:19.779144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:19.779144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:19.779144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:19.779144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:19.779144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:19.779144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:19.779144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:19.779144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:19.779144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:19.779144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:19.779144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:19.779144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:19.779144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:19.779144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:19.780025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:19.780025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:19.780025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:19.786553 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 341/668] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:19.787671 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a45bf9-f733-4673-9bc8-c3c7ad3385aa None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:19.803995 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f927bda-bcf1-4ee6-94f0-0d3dfbfc3a71 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:19.860237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a45e8-79af-4722-9795-72188a93d268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:19.861334 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a45e8-79af-4722-9795-72188a93d268 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.862858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a45e8-79af-4722-9795-72188a93d268 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.863310 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a45e8-79af-4722-9795-72188a93d268 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.874478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a45e8-79af-4722-9795-72188a93d268 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:19.876436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a45e8-79af-4722-9795-72188a93d268 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:19.893279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-180a45e8-79af-4722-9795-72188a93d268 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:19.893279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:19.893279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:19.893279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:19.893279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:19.893279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:19.893279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:19.893279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:19.893279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:19.893279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:19.893279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:19.893279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:19.893279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:19.893279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:19.893279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:19.894243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:19.894243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:19.894243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:19.894243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:19.894243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:19.894243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:19.894243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:19.894243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:19.894243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:19.894243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:19.894243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:19.894243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:19.894243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:19.894243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:19.894243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:19.894243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:19.894243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:19.895209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:19.895209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:19.895209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:19.895209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:19.895209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:19.895209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:19.895209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:19.895209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:19.898870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 328/669] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:19.900730 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a45e8-79af-4722-9795-72188a93d268 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:19.917840 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a45bf9-f733-4673-9bc8-c3c7ad3385aa None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:19.963194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07af9b-3c4d-4622-aac3-71cd5aafe91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:19.963545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07af9b-3c4d-4622-aac3-71cd5aafe91c None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:19.963761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07af9b-3c4d-4622-aac3-71cd5aafe91c None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:19.964032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07af9b-3c4d-4622-aac3-71cd5aafe91c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:19.973387 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07af9b-3c4d-4622-aac3-71cd5aafe91c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:19.974702 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07af9b-3c4d-4622-aac3-71cd5aafe91c None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:19.989186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-0c07af9b-3c4d-4622-aac3-71cd5aafe91c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:19.989186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:19.989186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:19.989186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:19.989186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:19.989186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:19.989186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:19.989186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:19.989186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:19.989186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:19.989186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:19.989186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:19.989186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:19.989186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:19.989186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:19.990298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:19.990298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:19.990298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:19.990298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:19.990298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:19.990298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:19.990298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:19.990298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:19.990298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:19.990298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:19.990298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:19.990298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:19.990298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:19.990298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:19.990298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:19.990298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:19.990298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:19.991349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:19.991349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:19.991349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:19.991349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:19.991349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:19.991349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:19.991349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:19.991349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:19.991349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 342/670] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:19.991349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07af9b-3c4d-4622-aac3-71cd5aafe91c None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.045787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a45e8-79af-4722-9795-72188a93d268 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.054605 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:20.073235 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:20.087838 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d50ff-023a-4f3e-acd1-3d3de7ec293c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.088497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d50ff-023a-4f3e-acd1-3d3de7ec293c None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.088726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d50ff-023a-4f3e-acd1-3d3de7ec293c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.089016 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d50ff-023a-4f3e-acd1-3d3de7ec293c None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.089651 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d50ff-023a-4f3e-acd1-3d3de7ec293c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:20.090076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d50ff-023a-4f3e-acd1-3d3de7ec293c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:20.091421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d50ff-023a-4f3e-acd1-3d3de7ec293c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:20.106448 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 329/671] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:20.107363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d50ff-023a-4f3e-acd1-3d3de7ec293c None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.107785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07af9b-3c4d-4622-aac3-71cd5aafe91c None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.145889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637ec18-9a64-440d-b10a-c65ad34b74d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.146505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637ec18-9a64-440d-b10a-c65ad34b74d7 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.146799 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637ec18-9a64-440d-b10a-c65ad34b74d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.147082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637ec18-9a64-440d-b10a-c65ad34b74d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.147775 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637ec18-9a64-440d-b10a-c65ad34b74d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:20.148082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637ec18-9a64-440d-b10a-c65ad34b74d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:20.149529 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637ec18-9a64-440d-b10a-c65ad34b74d7 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:20.161803 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 343/672] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:20.163755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637ec18-9a64-440d-b10a-c65ad34b74d7 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.215406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d50ff-023a-4f3e-acd1-3d3de7ec293c None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.253937 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f196de8-71fa-4454-8fe7-99b67c312ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.254556 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f196de8-71fa-4454-8fe7-99b67c312ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.254863 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f196de8-71fa-4454-8fe7-99b67c312ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.255126 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f196de8-71fa-4454-8fe7-99b67c312ce1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.265415 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f196de8-71fa-4454-8fe7-99b67c312ce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:20.270392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-2f196de8-71fa-4454-8fe7-99b67c312ce1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:20.270392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:20.270392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:20.270392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:20.270392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:20.270392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:20.270392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:20.270392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:20.270392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:20.270392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:20.270392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:20.270392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:20.270392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:20.270392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:20.270392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:20.270392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:20.271394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:20.271394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:20.271394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:20.271394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:20.271394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:20.271394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:20.271394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:20.271394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:20.271394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:20.271394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:20.271394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:20.271394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:20.271394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:20.271394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:20.271394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:20.271394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:20.272237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:20.272237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:20.272237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:20.272237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:20.272237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:20.272237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:20.272237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:20.272237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:20.272237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:20.272237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:20.272237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:20.272237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:20.272237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 330/673] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:20.272940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcf48c-6d53-4db2-9111-289feaa8b8b2 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.273007 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcf48c-6d53-4db2-9111-289feaa8b8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.273280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcf48c-6d53-4db2-9111-289feaa8b8b2 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.278537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 331/674] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:20.278537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcf48c-6d53-4db2-9111-289feaa8b8b2 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.284847 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637ec18-9a64-440d-b10a-c65ad34b74d7 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.302995 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:20.322910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65386c1-6628-4e4c-bb4f-3174cf78f4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.323521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65386c1-6628-4e4c-bb4f-3174cf78f4df None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.323796 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65386c1-6628-4e4c-bb4f-3174cf78f4df None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.324068 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65386c1-6628-4e4c-bb4f-3174cf78f4df None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.324750 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65386c1-6628-4e4c-bb4f-3174cf78f4df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:20.325057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65386c1-6628-4e4c-bb4f-3174cf78f4df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:20.325641 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:20.327397 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65386c1-6628-4e4c-bb4f-3174cf78f4df None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:20.339972 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 344/675] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:20.341185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65386c1-6628-4e4c-bb4f-3174cf78f4df None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.393593 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcf48c-6d53-4db2-9111-289feaa8b8b2 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.432293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9cfa7-d7ce-4cd4-aaaf-cc28e7da1c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.432916 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9cfa7-d7ce-4cd4-aaaf-cc28e7da1c0b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.433190 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9cfa7-d7ce-4cd4-aaaf-cc28e7da1c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.433457 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9cfa7-d7ce-4cd4-aaaf-cc28e7da1c0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.440158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9cfa7-d7ce-4cd4-aaaf-cc28e7da1c0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:20.441608 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9cfa7-d7ce-4cd4-aaaf-cc28e7da1c0b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:20.446302 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65386c1-6628-4e4c-bb4f-3174cf78f4df None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.452824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-a9f9cfa7-d7ce-4cd4-aaaf-cc28e7da1c0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:20.452824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:20.452824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:20.452824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:20.452824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:20.452824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:20.452824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:20.452824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:20.452824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:20.452824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:20.452824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:20.452824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:20.452824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:20.452824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:20.452824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:20.453775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:20.453775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:20.453775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:20.453775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:20.453775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:20.453775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:20.453775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:20.453775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:20.453775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:20.453775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:20.453775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:20.453775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:20.453775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:20.453775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:20.453775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:20.453775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:20.453775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:20.454668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:20.454668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:20.454668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:20.454668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:20.454668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:20.454668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:20.454668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:20.454668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:20.454668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 332/676] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:20.455228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9cfa7-d7ce-4cd4-aaaf-cc28e7da1c0b None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.494675 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a46f4c-6de5-40b2-805a-74ff3b207b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.494926 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a46f4c-6de5-40b2-805a-74ff3b207b8a None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.495213 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a46f4c-6de5-40b2-805a-74ff3b207b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.495361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a46f4c-6de5-40b2-805a-74ff3b207b8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.496094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a46f4c-6de5-40b2-805a-74ff3b207b8a None admin] RBAC: Adding query filter params (role.id=35bdd5122aae45ebbe44158a433d2f99, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=1d265d111b534944b7812cf067e317a2) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:20.496462 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a46f4c-6de5-40b2-805a-74ff3b207b8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:20.497925 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a46f4c-6de5-40b2-805a-74ff3b207b8a None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:20.504154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 345/677] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/role_assignments?user.id=1d265d111b534944b7812cf067e317a2&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=35bdd5122aae45ebbe44158a433d2f99 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:20.505337 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e55e4-fba7-4c3f-a070-17b8d1e0dfc4 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.505624 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e55e4-fba7-4c3f-a070-17b8d1e0dfc4 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.505900 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e55e4-fba7-4c3f-a070-17b8d1e0dfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.528057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-205e55e4-fba7-4c3f-a070-17b8d1e0dfc4 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:20.529319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-205e55e4-fba7-4c3f-a070-17b8d1e0dfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:20.579952 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:20.587261 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:20.590574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9cfa7-d7ce-4cd4-aaaf-cc28e7da1c0b None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.628886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 346/678] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:20.630217 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e55e4-fba7-4c3f-a070-17b8d1e0dfc4 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.635611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfe76d-594b-4cae-adcd-bfe5c7a811f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.636605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfe76d-594b-4cae-adcd-bfe5c7a811f3 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.637079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfe76d-594b-4cae-adcd-bfe5c7a811f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.637545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfe76d-594b-4cae-adcd-bfe5c7a811f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.638489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfe76d-594b-4cae-adcd-bfe5c7a811f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:20.639752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfe76d-594b-4cae-adcd-bfe5c7a811f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:20.642668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfe76d-594b-4cae-adcd-bfe5c7a811f3 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:20.655350 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 333/679] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:20.656699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfe76d-594b-4cae-adcd-bfe5c7a811f3 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.767929 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e55e4-fba7-4c3f-a070-17b8d1e0dfc4 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.799006 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfe76d-594b-4cae-adcd-bfe5c7a811f3 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.815075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e6397-5999-4c55-a491-94dd371f9976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.815690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e6397-5999-4c55-a491-94dd371f9976 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.815972 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e6397-5999-4c55-a491-94dd371f9976 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.816342 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e6397-5999-4c55-a491-94dd371f9976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.817075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e6397-5999-4c55-a491-94dd371f9976 None admin] RBAC: Adding query filter params (role.id=35bdd5122aae45ebbe44158a433d2f99, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=bb5248cb1c39412680cce0d9891fe1ff) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:20.817385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e6397-5999-4c55-a491-94dd371f9976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:20.818831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e6397-5999-4c55-a491-94dd371f9976 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:20.825294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 347/680] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/role_assignments?user.id=bb5248cb1c39412680cce0d9891fe1ff&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=35bdd5122aae45ebbe44158a433d2f99 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:20.826359 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e6397-5999-4c55-a491-94dd371f9976 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.855899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae89e23-34d9-4d24-9c76-d1d6c7cf2c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.856897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae89e23-34d9-4d24-9c76-d1d6c7cf2c87 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.857392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae89e23-34d9-4d24-9c76-d1d6c7cf2c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.857854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae89e23-34d9-4d24-9c76-d1d6c7cf2c87 None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:20.858821 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae89e23-34d9-4d24-9c76-d1d6c7cf2c87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:20.866760 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae89e23-34d9-4d24-9c76-d1d6c7cf2c87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:20.867189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:20.869503 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae89e23-34d9-4d24-9c76-d1d6c7cf2c87 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:20.901280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 334/681] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:20.907358 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae89e23-34d9-4d24-9c76-d1d6c7cf2c87 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:20.947029 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e6397-5999-4c55-a491-94dd371f9976 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:20.951303 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:20.990283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d678e22-68f1-4c72-9a6a-17647ab21ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:20.991402 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d678e22-68f1-4c72-9a6a-17647ab21ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:20.991893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d678e22-68f1-4c72-9a6a-17647ab21ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:20.992391 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d678e22-68f1-4c72-9a6a-17647ab21ad6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:21.005852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d678e22-68f1-4c72-9a6a-17647ab21ad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:21.007756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d678e22-68f1-4c72-9a6a-17647ab21ad6 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:21.024795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-3d678e22-68f1-4c72-9a6a-17647ab21ad6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:21.024795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:21.024795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:21.024795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:21.024795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:21.024795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:21.024795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:21.024795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:21.024795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:21.024795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:21.024795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:21.024795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:21.024795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:21.024795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:21.024795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:21.024795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:21.025777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:21.025777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:21.025777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:21.025777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:21.025777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:21.025777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:21.025777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:21.025777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:21.025777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:21.025777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:21.025777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:21.025777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:21.025777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:21.025777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:21.025777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:21.025777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:21.025777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:21.026658 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:21.026658 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:21.026658 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:21.029070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 348/682] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:21.030262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d678e22-68f1-4c72-9a6a-17647ab21ad6 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:21.041866 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae89e23-34d9-4d24-9c76-d1d6c7cf2c87 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:21.087769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49efb76-b6e3-44c8-b71b-215ac2616768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:21.087978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49efb76-b6e3-44c8-b71b-215ac2616768 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:21.088239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49efb76-b6e3-44c8-b71b-215ac2616768 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:21.088523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49efb76-b6e3-44c8-b71b-215ac2616768 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:21.089592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49efb76-b6e3-44c8-b71b-215ac2616768 None admin] RBAC: Adding query filter params (role.id=35bdd5122aae45ebbe44158a433d2f99, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=f77bbd8de87042aab3674db366635cf5) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:21.090453 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49efb76-b6e3-44c8-b71b-215ac2616768 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:21.091619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49efb76-b6e3-44c8-b71b-215ac2616768 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:21.099302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 335/683] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:20 2022] GET /identity/v3/role_assignments?user.id=f77bbd8de87042aab3674db366635cf5&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=35bdd5122aae45ebbe44158a433d2f99 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:21.100166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49efb76-b6e3-44c8-b71b-215ac2616768 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:21.151265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:21.182428 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d678e22-68f1-4c72-9a6a-17647ab21ad6 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:21.226538 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:21.231757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c59642-7c77-4f97-9048-8432fe1ff48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:21.232411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c59642-7c77-4f97-9048-8432fe1ff48e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:21.232688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c59642-7c77-4f97-9048-8432fe1ff48e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:21.232957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c59642-7c77-4f97-9048-8432fe1ff48e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:21.247361 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49efb76-b6e3-44c8-b71b-215ac2616768 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:21.248582 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c59642-7c77-4f97-9048-8432fe1ff48e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:21.254607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-d8c59642-7c77-4f97-9048-8432fe1ff48e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:21.254607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:21.254607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:21.254607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:21.254607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:21.254607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:21.254607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:21.254607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:21.254607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:21.254607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:21.254607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:21.254607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:21.254607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:21.254607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:21.254607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:21.254607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:21.255653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:21.255653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:21.255653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:21.255653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:21.255653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:21.255653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:21.255653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:21.255653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:21.255653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:21.255653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:21.255653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:21.255653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:21.255653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:21.255653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:21.255653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:21.255653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:21.256944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:21.256944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:21.256944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:21.256944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:21.256944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:21.256944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:21.256944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:21.256944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:21.256944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:21.256944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:21.256944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:21.256944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:21.256944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 349/684] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:21.262833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c59642-7c77-4f97-9048-8432fe1ff48e None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:21.293184 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8752b-78f5-4173-9c28-0f306f01d13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:21.294283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8752b-78f5-4173-9c28-0f306f01d13b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:21.294844 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8752b-78f5-4173-9c28-0f306f01d13b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:21.295384 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8752b-78f5-4173-9c28-0f306f01d13b None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:21.296224 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8752b-78f5-4173-9c28-0f306f01d13b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:21.296930 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8752b-78f5-4173-9c28-0f306f01d13b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:21.299233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8752b-78f5-4173-9c28-0f306f01d13b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:21.314767 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 336/685] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:21.330710 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8752b-78f5-4173-9c28-0f306f01d13b None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:21.398143 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c59642-7c77-4f97-9048-8432fe1ff48e None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:21.438949 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f27760-2878-4ba7-98e8-2f0aab239823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:21.439703 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f27760-2878-4ba7-98e8-2f0aab239823 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:21.440076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f27760-2878-4ba7-98e8-2f0aab239823 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:21.440441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f27760-2878-4ba7-98e8-2f0aab239823 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:21.441205 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f27760-2878-4ba7-98e8-2f0aab239823 None admin] RBAC: Adding query filter params (name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:21.441607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f27760-2878-4ba7-98e8-2f0aab239823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:21.443920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f27760-2878-4ba7-98e8-2f0aab239823 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:21.455023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 350/686] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:21.460718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f27760-2878-4ba7-98e8-2f0aab239823 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:21.474252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8752b-78f5-4173-9c28-0f306f01d13b None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:21.475518 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:21.512847 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00c8a2-d8e6-45e9-b8e1-6150d2b99dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:21.514211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00c8a2-d8e6-45e9-b8e1-6150d2b99dff None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:21.514645 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00c8a2-d8e6-45e9-b8e1-6150d2b99dff None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:21.515093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00c8a2-d8e6-45e9-b8e1-6150d2b99dff None admin] PATH_INFO: `/v3/users/nova` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:21.528637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00c8a2-d8e6-45e9-b8e1-6150d2b99dff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:21.530663 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00c8a2-d8e6-45e9-b8e1-6150d2b99dff None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:21.544425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-4d00c8a2-d8e6-45e9-b8e1-6150d2b99dff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:21.544425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:21.544425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:21.544425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:21.544425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:21.544425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:21.544425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:21.544425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:21.544425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:21.544425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:21.544425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:21.544425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:21.544425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:21.544425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:21.544425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:21.544425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:21.545575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:21.546759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:21.546759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:21.546759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:21.546759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:21.546759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:21.546759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:21.546759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:21.546759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:21.546759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:21.546759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:21.549589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 337/687] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Sep 22 14:45:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:21.555141 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00c8a2-d8e6-45e9-b8e1-6150d2b99dff None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:21.571065 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:21.626663 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f27760-2878-4ba7-98e8-2f0aab239823 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:21.694571 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00c8a2-d8e6-45e9-b8e1-6150d2b99dff None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:21.699406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08764a3-be53-4092-97c6-769a51aa8742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:21.700049 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08764a3-be53-4092-97c6-769a51aa8742 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:21.700321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08764a3-be53-4092-97c6-769a51aa8742 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:21.700589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08764a3-be53-4092-97c6-769a51aa8742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:21.725783 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08764a3-be53-4092-97c6-769a51aa8742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:21.727287 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08764a3-be53-4092-97c6-769a51aa8742 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:21.730554 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510f655-cbcc-45f0-aaa2-7b008b9bfca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:21.731306 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510f655-cbcc-45f0-aaa2-7b008b9bfca0 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:21.731583 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510f655-cbcc-45f0-aaa2-7b008b9bfca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:21.731853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510f655-cbcc-45f0-aaa2-7b008b9bfca0 None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:21.732585 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510f655-cbcc-45f0-aaa2-7b008b9bfca0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:21.732886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510f655-cbcc-45f0-aaa2-7b008b9bfca0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:21.734357 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510f655-cbcc-45f0-aaa2-7b008b9bfca0 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:21.753801 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 338/688] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Sep 22 14:45:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:21.756100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-f08764a3-be53-4092-97c6-769a51aa8742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:21.756100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:21.756100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:21.756100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:21.756100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:21.756100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:21.756100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:21.756100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:21.756100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:21.756100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:21.756100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:21.756100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:21.756100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:21.756100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:21.756100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:21.756100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:21.757061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:21.757061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:21.757061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:21.757061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:21.757061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:21.757061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:21.757061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:21.757061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:21.757061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:21.757061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:21.757061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:21.757061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:21.757061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:21.757061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:21.757061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:21.757061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:21.757061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:21.758528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:21.758528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:21.758528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:21.758528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 351/689] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:21.759757 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510f655-cbcc-45f0-aaa2-7b008b9bfca0 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:21.764849 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08764a3-be53-4092-97c6-769a51aa8742 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:21.853562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:21.867455 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:21.929602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510f655-cbcc-45f0-aaa2-7b008b9bfca0 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:21.962530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08764a3-be53-4092-97c6-769a51aa8742 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:21.968416 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6601f17-5ee5-4af4-9bad-37b536ad39ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:21.969116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6601f17-5ee5-4af4-9bad-37b536ad39ff None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:21.969397 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6601f17-5ee5-4af4-9bad-37b536ad39ff None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:21.969664 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6601f17-5ee5-4af4-9bad-37b536ad39ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:22.031927 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609c47f-1b7b-4d9c-b354-a0195c06cbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:22.032545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609c47f-1b7b-4d9c-b354-a0195c06cbe4 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:22.032821 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609c47f-1b7b-4d9c-b354-a0195c06cbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:22.033089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609c47f-1b7b-4d9c-b354-a0195c06cbe4 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:22.033726 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609c47f-1b7b-4d9c-b354-a0195c06cbe4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:22.034023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609c47f-1b7b-4d9c-b354-a0195c06cbe4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:22.035415 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609c47f-1b7b-4d9c-b354-a0195c06cbe4 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:22.063784 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 352/690] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:22.072143 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609c47f-1b7b-4d9c-b354-a0195c06cbe4 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:22.078151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6601f17-5ee5-4af4-9bad-37b536ad39ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:22.080459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6601f17-5ee5-4af4-9bad-37b536ad39ff None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:22.197512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:22.271943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609c47f-1b7b-4d9c-b354-a0195c06cbe4 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:22.272321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 339/691] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 14:45:22.274871 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6601f17-5ee5-4af4-9bad-37b536ad39ff None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:22.322814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:22.332852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8c409-0ea6-4a90-b403-c261d5a50d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:22.333620 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8c409-0ea6-4a90-b403-c261d5a50d03 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:22.334012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8c409-0ea6-4a90-b403-c261d5a50d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:22.334383 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8c409-0ea6-4a90-b403-c261d5a50d03 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:22.342081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8c409-0ea6-4a90-b403-c261d5a50d03 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:22.343937 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8c409-0ea6-4a90-b403-c261d5a50d03 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:22.357771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-2cf8c409-0ea6-4a90-b403-c261d5a50d03 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:22.357771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:22.357771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:22.357771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:22.357771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:22.357771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:22.357771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:22.357771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:22.357771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:22.357771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:22.357771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:22.357771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:22.357771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:22.357771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:22.357771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:22.357771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:22.358970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:22.360676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:22.360676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:22.360676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:22.360676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:22.360676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:22.360676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:22.360676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:22.360676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:22.360676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:22.360676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:22.366787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 353/692] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:22.374768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8c409-0ea6-4a90-b403-c261d5a50d03 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:22.478203 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8c409-0ea6-4a90-b403-c261d5a50d03 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:22.515668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9c571-6aba-4e92-9cd9-ff52975aa359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:22.516483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9c571-6aba-4e92-9cd9-ff52975aa359 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:22.517312 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9c571-6aba-4e92-9cd9-ff52975aa359 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:22.517688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9c571-6aba-4e92-9cd9-ff52975aa359 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:22.518666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9c571-6aba-4e92-9cd9-ff52975aa359 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:22.519226 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9c571-6aba-4e92-9cd9-ff52975aa359 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:22.520885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9c571-6aba-4e92-9cd9-ff52975aa359 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:22.524395 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6601f17-5ee5-4af4-9bad-37b536ad39ff None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:22.546619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 354/693] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Sep 22 14:45:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:22.548467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9c571-6aba-4e92-9cd9-ff52975aa359 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:22.597212 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:22.631247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e73426-cef5-4eb4-bb36-7d82fc0c7ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:22.632295 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e73426-cef5-4eb4-bb36-7d82fc0c7ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:22.634541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e73426-cef5-4eb4-bb36-7d82fc0c7ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:22.634994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e73426-cef5-4eb4-bb36-7d82fc0c7ad0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:22.654670 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e73426-cef5-4eb4-bb36-7d82fc0c7ad0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:22.658619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e73426-cef5-4eb4-bb36-7d82fc0c7ad0 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:22.685725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-44e73426-cef5-4eb4-bb36-7d82fc0c7ad0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:22.685725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:22.685725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:22.685725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:22.685725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:22.685725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:22.685725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:22.685725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:22.685725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:22.685725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:22.685725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:22.685725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:22.685725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:22.685725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:22.685725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:22.685725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:22.686945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:22.686945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:22.686945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:22.686945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:22.686945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:22.686945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:22.686945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:22.686945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:22.686945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:22.686945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:22.686945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:22.686945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:22.686945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:22.686945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:22.686945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:22.686945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:22.686945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:22.688554 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:22.688554 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:22.688554 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:22.691512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 340/694] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:22.696859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e73426-cef5-4eb4-bb36-7d82fc0c7ad0 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:22.706419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9c571-6aba-4e92-9cd9-ff52975aa359 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:22.734365 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:22.779746 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e66f5a-f471-4d02-9925-47ac99443452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:22.780362 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e66f5a-f471-4d02-9925-47ac99443452 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:22.780640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e66f5a-f471-4d02-9925-47ac99443452 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:22.780909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e66f5a-f471-4d02-9925-47ac99443452 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:22.881221 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e66f5a-f471-4d02-9925-47ac99443452 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:22.884122 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e66f5a-f471-4d02-9925-47ac99443452 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:22.893155 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e73426-cef5-4eb4-bb36-7d82fc0c7ad0 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:22.957567 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857222f1-f22a-4e52-9b24-3cbd2c29264e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:22.958183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857222f1-f22a-4e52-9b24-3cbd2c29264e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:22.958462 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857222f1-f22a-4e52-9b24-3cbd2c29264e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:22.958747 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857222f1-f22a-4e52-9b24-3cbd2c29264e None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:22.959386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857222f1-f22a-4e52-9b24-3cbd2c29264e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:22.959685 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857222f1-f22a-4e52-9b24-3cbd2c29264e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:22.961917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857222f1-f22a-4e52-9b24-3cbd2c29264e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:22.967193 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:22.991258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 341/695] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:22.996702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857222f1-f22a-4e52-9b24-3cbd2c29264e None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:23.060102 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 355/696] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 14:45:23.061392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e66f5a-f471-4d02-9925-47ac99443452 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:23.157729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:23.185015 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857222f1-f22a-4e52-9b24-3cbd2c29264e None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:23.223403 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1df50-4ae0-43ab-a2e1-670a61b0f245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:23.224458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1df50-4ae0-43ab-a2e1-670a61b0f245 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.224874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e66f5a-f471-4d02-9925-47ac99443452 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:23.225194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1df50-4ae0-43ab-a2e1-670a61b0f245 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.225677 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1df50-4ae0-43ab-a2e1-670a61b0f245 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.240731 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1df50-4ae0-43ab-a2e1-670a61b0f245 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:23.242563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1df50-4ae0-43ab-a2e1-670a61b0f245 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:23.266215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-5ed1df50-4ae0-43ab-a2e1-670a61b0f245 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:23.266215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:23.266215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:23.266215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:23.266215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:23.266215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:23.266215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:23.266215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:23.266215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:23.266215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:23.266215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:23.266215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:23.266215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:23.266215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:23.266215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:23.267379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:23.267379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:23.267379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:23.267379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:23.267379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:23.267379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:23.267379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:23.267379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:23.267379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:23.267379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:23.267379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:23.267379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:23.267379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:23.267379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:23.267379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:23.267379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:23.267379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:23.268386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:23.268386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:23.268386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:23.268386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:23.268386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:23.268386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:23.268386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:23.268386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:23.268386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 342/697] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:23.270027 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b376ff7-d97e-49cd-aad4-bc6af57ea019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:23.270993 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b376ff7-d97e-49cd-aad4-bc6af57ea019 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.271483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b376ff7-d97e-49cd-aad4-bc6af57ea019 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.271959 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b376ff7-d97e-49cd-aad4-bc6af57ea019 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.273516 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1df50-4ae0-43ab-a2e1-670a61b0f245 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:23.289916 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:23.296737 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b376ff7-d97e-49cd-aad4-bc6af57ea019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:23.298948 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b376ff7-d97e-49cd-aad4-bc6af57ea019 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:23.320442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-7b376ff7-d97e-49cd-aad4-bc6af57ea019 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:23.320442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:23.320442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:23.320442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:23.320442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:23.320442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:23.320442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:23.320442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:23.320442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:23.320442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:23.320442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:23.320442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:23.320442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:23.320442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:23.320442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:23.320442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:23.321495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:23.321495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:23.321495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:23.321495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:23.321495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:23.321495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:23.321495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:23.321495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:23.321495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:23.321495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:23.321495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:23.321495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:23.321495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:23.321495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:23.321495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:23.321495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:23.321495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:23.322374 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:23.322374 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:23.322374 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:23.327880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 356/698] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:23.329228 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0c13d-9943-4f9d-94dd-f38578a7e226 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.329516 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0c13d-9943-4f9d-94dd-f38578a7e226 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.329792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0c13d-9943-4f9d-94dd-f38578a7e226 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.335385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 357/699] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:23.335385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0c13d-9943-4f9d-94dd-f38578a7e226 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:23.428141 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1df50-4ae0-43ab-a2e1-670a61b0f245 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:23.465181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0c13d-9943-4f9d-94dd-f38578a7e226 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:23.492303 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f23046-38e0-4e65-b435-54eba8ef7acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:23.493649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f23046-38e0-4e65-b435-54eba8ef7acf None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.494432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f23046-38e0-4e65-b435-54eba8ef7acf None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.494966 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f23046-38e0-4e65-b435-54eba8ef7acf None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.496581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f23046-38e0-4e65-b435-54eba8ef7acf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:23.497171 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f23046-38e0-4e65-b435-54eba8ef7acf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:23.499868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f23046-38e0-4e65-b435-54eba8ef7acf None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:23.504775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:23.507534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384d440-6c22-4561-8e59-49c1888e5d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:23.508273 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384d440-6c22-4561-8e59-49c1888e5d01 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.508659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384d440-6c22-4561-8e59-49c1888e5d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.509054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384d440-6c22-4561-8e59-49c1888e5d01 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.509804 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384d440-6c22-4561-8e59-49c1888e5d01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:23.510218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384d440-6c22-4561-8e59-49c1888e5d01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:23.511793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384d440-6c22-4561-8e59-49c1888e5d01 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:23.523300 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 343/700] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:23.525191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31191591-039e-4952-9241-15dd1cf30503 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.525191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31191591-039e-4952-9241-15dd1cf30503 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.525191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31191591-039e-4952-9241-15dd1cf30503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.533003 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 358/701] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:23.534196 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384d440-6c22-4561-8e59-49c1888e5d01 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:23.552624 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-31191591-039e-4952-9241-15dd1cf30503 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:23.553868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-31191591-039e-4952-9241-15dd1cf30503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:23.601658 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:23.634776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 344/702] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:23.634776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31191591-039e-4952-9241-15dd1cf30503 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:23.658979 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384d440-6c22-4561-8e59-49c1888e5d01 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:23.696422 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b49da-18bb-44ba-996f-ed408aa84e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:23.697032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b49da-18bb-44ba-996f-ed408aa84e47 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.697312 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b49da-18bb-44ba-996f-ed408aa84e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.697583 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b49da-18bb-44ba-996f-ed408aa84e47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.698297 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b49da-18bb-44ba-996f-ed408aa84e47 None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=0c9be41c39494e54a14398a1f03c4d69) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:23.698603 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b49da-18bb-44ba-996f-ed408aa84e47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:23.700078 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b49da-18bb-44ba-996f-ed408aa84e47 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:23.705667 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 359/703] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:23 2022] GET /identity/v3/role_assignments?user.id=0c9be41c39494e54a14398a1f03c4d69&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=32c8a81e339a462e973e76bb528ef35a => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:23.706680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72730f0-a02e-4691-8564-1edf963e2a66 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.707044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72730f0-a02e-4691-8564-1edf963e2a66 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.707263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72730f0-a02e-4691-8564-1edf963e2a66 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.708729 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 360/704] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:23.709775 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b6254-46de-4589-93d9-2d26c69efd86 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.710054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b6254-46de-4589-93d9-2d26c69efd86 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.710328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b6254-46de-4589-93d9-2d26c69efd86 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.717056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 361/705] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:23.718076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf66d6-d91f-4678-b30a-9f6ecd22e5ce None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.718363 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf66d6-d91f-4678-b30a-9f6ecd22e5ce None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.718774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf66d6-d91f-4678-b30a-9f6ecd22e5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-caaf66d6-d91f-4678-b30a-9f6ecd22e5ce None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:23.740819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-caaf66d6-d91f-4678-b30a-9f6ecd22e5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:23.781917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31191591-039e-4952-9241-15dd1cf30503 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:23.820857 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 362/706] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:23.820857 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b956a7-adfe-4750-b60a-ade7a3cca107 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.820857 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b956a7-adfe-4750-b60a-ade7a3cca107 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.820857 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b956a7-adfe-4750-b60a-ade7a3cca107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.828258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:23.849775 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-73b956a7-adfe-4750-b60a-ade7a3cca107 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:23.852213 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-73b956a7-adfe-4750-b60a-ade7a3cca107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:23.856951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a35b0-1065-48fd-8f05-7967c4927f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:23.857462 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a35b0-1065-48fd-8f05-7967c4927f58 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:23.857788 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a35b0-1065-48fd-8f05-7967c4927f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:23.858061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a35b0-1065-48fd-8f05-7967c4927f58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:23.872841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a35b0-1065-48fd-8f05-7967c4927f58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:23.874387 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a35b0-1065-48fd-8f05-7967c4927f58 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:23.891445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-2b9a35b0-1065-48fd-8f05-7967c4927f58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:23.891445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:23.891445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:23.891445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:23.891445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:23.891445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:23.891445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:23.891445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:23.891445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:23.891445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:23.891445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:23.891445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:23.891445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:23.891445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:23.891445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:23.892417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:23.892417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:23.892417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:23.892417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:23.892417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:23.892417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:23.892417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:23.892417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:23.892417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:23.892417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:23.892417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:23.892417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:23.892417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:23.892417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:23.892417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:23.892417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:23.892417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:23.893774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:23.893774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:23.893774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:23.893774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:23.893774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:23.893774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:23.893774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:23.893774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:23.893774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 345/707] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:23.899802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a35b0-1065-48fd-8f05-7967c4927f58 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:23.927170 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:23.970814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 363/708] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:23.972114 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b956a7-adfe-4750-b60a-ade7a3cca107 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:24.018813 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a35b0-1065-48fd-8f05-7967c4927f58 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:24.062251 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b10131-e0de-4f6b-b0d0-976a28a9014d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:24.062852 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b10131-e0de-4f6b-b0d0-976a28a9014d None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.064938 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b10131-e0de-4f6b-b0d0-976a28a9014d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.065221 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b10131-e0de-4f6b-b0d0-976a28a9014d None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.065923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b10131-e0de-4f6b-b0d0-976a28a9014d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:24.066232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b10131-e0de-4f6b-b0d0-976a28a9014d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:24.067703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b10131-e0de-4f6b-b0d0-976a28a9014d None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:24.097242 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 346/709] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:24.103363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b10131-e0de-4f6b-b0d0-976a28a9014d None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:24.178394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:24.199776 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b956a7-adfe-4750-b60a-ade7a3cca107 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:24.253374 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe45bd7-39c0-426e-b1d9-51169b347886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:24.254848 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe45bd7-39c0-426e-b1d9-51169b347886 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.255321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe45bd7-39c0-426e-b1d9-51169b347886 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.255784 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe45bd7-39c0-426e-b1d9-51169b347886 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.256573 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe45bd7-39c0-426e-b1d9-51169b347886 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:24.258186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe45bd7-39c0-426e-b1d9-51169b347886 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:24.260110 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b10131-e0de-4f6b-b0d0-976a28a9014d None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:24.270835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-6fe45bd7-39c0-426e-b1d9-51169b347886 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:24.270835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:24.270835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:24.270835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:24.270835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:24.270835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:24.270835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:24.270835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:24.270835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:24.270835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:24.270835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:24.270835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:24.270835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:24.270835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:24.270835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:24.270835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:24.270835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:24.271880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:24.271880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:24.271880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:24.271880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:24.271880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:24.271880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:24.271880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:24.271880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:24.271880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:24.271880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:24.271880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:24.271880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:24.271880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:24.271880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:24.271880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:24.271880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:24.272748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:24.272748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:24.272748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:24.272748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:24.272748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:24.272748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:24.272748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:24.272748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:24.272748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:24.272748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:24.272748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:24.272748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:24.272748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:24.272748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:24.276489 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 364/710] 10.222.0.26 () {56 vars in 1129 bytes} [Thu Sep 22 14:45:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:24.276489 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe45bd7-39c0-426e-b1d9-51169b347886 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:24.301759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5090d-932a-47e5-a2e6-af504e37c5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:24.302376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5090d-932a-47e5-a2e6-af504e37c5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.302655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5090d-932a-47e5-a2e6-af504e37c5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.302959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5090d-932a-47e5-a2e6-af504e37c5a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.303683 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5090d-932a-47e5-a2e6-af504e37c5a9 None admin] RBAC: Adding query filter params (role.id=35bdd5122aae45ebbe44158a433d2f99, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=0a265623ae734936a340c97df852e227) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:24.303994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5090d-932a-47e5-a2e6-af504e37c5a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:24.305433 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5090d-932a-47e5-a2e6-af504e37c5a9 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:24.323225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 347/711] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:24 2022] GET /identity/v3/role_assignments?user.id=0a265623ae734936a340c97df852e227&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=35bdd5122aae45ebbe44158a433d2f99 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:24.324005 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5090d-932a-47e5-a2e6-af504e37c5a9 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:24.329640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:24.405736 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe45bd7-39c0-426e-b1d9-51169b347886 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:24.451046 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae7308-2aae-4f6c-a952-9a6b96d1e610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:24.451904 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae7308-2aae-4f6c-a952-9a6b96d1e610 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.452253 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae7308-2aae-4f6c-a952-9a6b96d1e610 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.452590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae7308-2aae-4f6c-a952-9a6b96d1e610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.467651 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5090d-932a-47e5-a2e6-af504e37c5a9 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:24.468806 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae7308-2aae-4f6c-a952-9a6b96d1e610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:24.470584 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae7308-2aae-4f6c-a952-9a6b96d1e610 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:24.481427 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:24.491326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-69ae7308-2aae-4f6c-a952-9a6b96d1e610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:24.491326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:24.491326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:24.491326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:24.491326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:24.491326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:24.491326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:24.491326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:24.491326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:24.491326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:24.491326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:24.491326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:24.491326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:24.491326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:24.491326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:24.491326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:24.492434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:24.492434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:24.492434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:24.492434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:24.492434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:24.492434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:24.492434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:24.492434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:24.492434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:24.492434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:24.492434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:24.492434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:24.492434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:24.492434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:24.492434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:24.492434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:24.492434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:24.493384 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:24.493384 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:24.493384 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:24.494881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 365/712] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:24.495985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae7308-2aae-4f6c-a952-9a6b96d1e610 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:24.522084 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4718291a-747c-4d12-a915-e198e7708ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:24.523051 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718291a-747c-4d12-a915-e198e7708ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.523480 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718291a-747c-4d12-a915-e198e7708ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.524343 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718291a-747c-4d12-a915-e198e7708ea4 None admin] PATH_INFO: `/v3/services/network` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.525562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4718291a-747c-4d12-a915-e198e7708ea4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:24.527238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4718291a-747c-4d12-a915-e198e7708ea4 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:24.544927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-4718291a-747c-4d12-a915-e198e7708ea4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:24.544927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:24.544927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:24.544927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:24.544927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:24.544927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:24.544927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:24.544927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:24.544927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:24.544927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:24.544927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:24.544927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:24.544927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:24.544927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:24.544927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:24.544927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:24.544927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:24.546265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:24.546265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:24.546265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:24.546265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:24.546265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:24.546265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:24.546265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:24.546265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:24.546265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:24.546265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:24.546265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:24.546265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:24.546265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:24.546265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:24.546265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:24.546265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:24.547824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:24.547824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:24.547824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:24.547824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:24.547824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:24.547824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:24.547824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:24.547824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:24.547824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:24.547824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:24.547824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:24.547824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:24.547824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:24.547824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:24.550296 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 348/713] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:24 2022] GET /identity/v3/services/network => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:24.551304 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4718291a-747c-4d12-a915-e198e7708ea4 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:24.632422 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:24.650109 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae7308-2aae-4f6c-a952-9a6b96d1e610 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:24.691307 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3fc57-faf0-4e0d-b555-4d7379f450d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:24.691914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3fc57-faf0-4e0d-b555-4d7379f450d1 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.692195 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3fc57-faf0-4e0d-b555-4d7379f450d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.692468 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3fc57-faf0-4e0d-b555-4d7379f450d1 None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.693099 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3fc57-faf0-4e0d-b555-4d7379f450d1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:24.693402 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3fc57-faf0-4e0d-b555-4d7379f450d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:24.694655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4718291a-747c-4d12-a915-e198e7708ea4 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:24.694885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3fc57-faf0-4e0d-b555-4d7379f450d1 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:24.700909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 366/714] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Sep 22 14:45:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:24.701773 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3fc57-faf0-4e0d-b555-4d7379f450d1 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:24.745747 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666b90f-58cc-465b-b55a-f60913a3f9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:24.746368 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666b90f-58cc-465b-b55a-f60913a3f9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.746646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666b90f-58cc-465b-b55a-f60913a3f9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.746941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666b90f-58cc-465b-b55a-f60913a3f9d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.747585 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666b90f-58cc-465b-b55a-f60913a3f9d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:24.747882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666b90f-58cc-465b-b55a-f60913a3f9d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:24.749828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666b90f-58cc-465b-b55a-f60913a3f9d9 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:24.767201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 349/715] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:24.768855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666b90f-58cc-465b-b55a-f60913a3f9d9 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:24.813565 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:24.820060 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3fc57-faf0-4e0d-b555-4d7379f450d1 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:24.861288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10496fe9-e04f-42b6-aed3-ad25b12f6625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:24.861585 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10496fe9-e04f-42b6-aed3-ad25b12f6625 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.861868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10496fe9-e04f-42b6-aed3-ad25b12f6625 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.862141 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10496fe9-e04f-42b6-aed3-ad25b12f6625 None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.862837 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10496fe9-e04f-42b6-aed3-ad25b12f6625 None admin] RBAC: Adding query filter params (name=network) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:24.863244 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10496fe9-e04f-42b6-aed3-ad25b12f6625 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:24.864594 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10496fe9-e04f-42b6-aed3-ad25b12f6625 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:24.878824 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 367/716] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:24.880400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10496fe9-e04f-42b6-aed3-ad25b12f6625 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:24.908162 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:24.925864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666b90f-58cc-465b-b55a-f60913a3f9d9 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:24.976463 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d369644-3579-4858-95c2-f847b0bc1694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:24.977349 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d369644-3579-4858-95c2-f847b0bc1694 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:24.977627 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d369644-3579-4858-95c2-f847b0bc1694 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:24.977895 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d369644-3579-4858-95c2-f847b0bc1694 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:24.979471 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d369644-3579-4858-95c2-f847b0bc1694 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:24.979784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d369644-3579-4858-95c2-f847b0bc1694 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:24.981157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d369644-3579-4858-95c2-f847b0bc1694 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:24.988874 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 350/717] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Sep 22 14:45:24 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:24.990277 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d369644-3579-4858-95c2-f847b0bc1694 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:25.019404 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10496fe9-e04f-42b6-aed3-ad25b12f6625 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:25.061834 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13918eb1-92d9-43b9-aa0a-9e2f7ffe6c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:25.063470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13918eb1-92d9-43b9-aa0a-9e2f7ffe6c05 None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:25.063962 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13918eb1-92d9-43b9-aa0a-9e2f7ffe6c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:25.064463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13918eb1-92d9-43b9-aa0a-9e2f7ffe6c05 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:25.065646 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13918eb1-92d9-43b9-aa0a-9e2f7ffe6c05 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:25.067411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13918eb1-92d9-43b9-aa0a-9e2f7ffe6c05 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:25.107070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 368/718] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:45:24 2022] POST /identity/v3/users => generated 254 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:25.127046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:25.131157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d369644-3579-4858-95c2-f847b0bc1694 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:25.173332 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17006b-9cba-4d6e-beea-31aacaa0bb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:25.173876 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17006b-9cba-4d6e-beea-31aacaa0bb81 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:25.174149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17006b-9cba-4d6e-beea-31aacaa0bb81 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:25.174418 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17006b-9cba-4d6e-beea-31aacaa0bb81 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:25.175087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17006b-9cba-4d6e-beea-31aacaa0bb81 None admin] RBAC: Adding query filter params (type=network) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:25.175552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17006b-9cba-4d6e-beea-31aacaa0bb81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:25.176955 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17006b-9cba-4d6e-beea-31aacaa0bb81 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:25.182973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 351/719] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:24 2022] GET /identity/v3/services?type=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:26.141830 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c008208-efc2-4b32-ab1a-d161da8a329f None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:26.142360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c008208-efc2-4b32-ab1a-d161da8a329f None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:26.142360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c008208-efc2-4b32-ab1a-d161da8a329f None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:26.144154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 369/720] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:26.149871 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335196ac-9ebe-4dc8-a924-589255a8294d None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:26.150395 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335196ac-9ebe-4dc8-a924-589255a8294d None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:26.150395 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335196ac-9ebe-4dc8-a924-589255a8294d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:26.170148 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-335196ac-9ebe-4dc8-a924-589255a8294d None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:26.171405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-335196ac-9ebe-4dc8-a924-589255a8294d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:26.259445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 352/721] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:26.688246 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c008208-efc2-4b32-ab1a-d161da8a329f None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:26.781073 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:26.817727 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c008208-efc2-4b32-ab1a-d161da8a329f None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:26.860048 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bbd50-743a-4560-847b-fab8ad4ec1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:26.860685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bbd50-743a-4560-847b-fab8ad4ec1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:26.860970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bbd50-743a-4560-847b-fab8ad4ec1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:26.861239 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bbd50-743a-4560-847b-fab8ad4ec1b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:26.878946 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33bbd50-743a-4560-847b-fab8ad4ec1b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:26.884443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-c33bbd50-743a-4560-847b-fab8ad4ec1b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:26.884443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:26.884443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:26.884443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:26.884443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:26.884443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:26.884443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:26.884443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:26.884443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:26.884443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:26.884443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:26.884443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:26.884443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:26.884443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:26.884443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:26.884443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:26.885400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:26.885400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:26.885400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:26.885400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:26.885400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:26.885400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:26.885400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:26.885400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:26.885400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:26.885400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:26.885400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:26.885400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:26.885400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:26.885400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:26.885400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:26.885400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:26.886241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:26.886241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:26.886241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:26.886241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:26.886241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:26.886241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:26.886241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:26.886241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:26.886241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:26.886241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:26.886241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:26.886241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:26.886241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 370/722] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:26.891698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335196ac-9ebe-4dc8-a924-589255a8294d None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:27.033344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:27.044693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335196ac-9ebe-4dc8-a924-589255a8294d None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:27.099216 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e25f6-31f0-41c6-9b43-1b41b62ff444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:27.099841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e25f6-31f0-41c6-9b43-1b41b62ff444 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.100122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e25f6-31f0-41c6-9b43-1b41b62ff444 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.100395 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e25f6-31f0-41c6-9b43-1b41b62ff444 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.101065 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e25f6-31f0-41c6-9b43-1b41b62ff444 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:27.101370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e25f6-31f0-41c6-9b43-1b41b62ff444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:27.102766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e25f6-31f0-41c6-9b43-1b41b62ff444 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:27.112490 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 353/723] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:27.117983 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bbd50-743a-4560-847b-fab8ad4ec1b9 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:27.228750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29406fad-1191-4eda-a771-452962cc814f None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.229262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29406fad-1191-4eda-a771-452962cc814f None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.229262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29406fad-1191-4eda-a771-452962cc814f None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.230885 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 354/724] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:27.232238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bbd50-743a-4560-847b-fab8ad4ec1b9 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:27.236140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5de416-06ff-406b-a406-446334ae0ee3 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.236419 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5de416-06ff-406b-a406-446334ae0ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.236698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5de416-06ff-406b-a406-446334ae0ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.274545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac4299-b32b-45ae-9620-f2d411e07e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:27.275697 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac4299-b32b-45ae-9620-f2d411e07e39 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.275697 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac4299-b32b-45ae-9620-f2d411e07e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.275697 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac4299-b32b-45ae-9620-f2d411e07e39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.276400 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-7f5de416-06ff-406b-a406-446334ae0ee3 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:27.281683 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7f5de416-06ff-406b-a406-446334ae0ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:27.283025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:27.299526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac4299-b32b-45ae-9620-f2d411e07e39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:27.300901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac4299-b32b-45ae-9620-f2d411e07e39 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:27.338467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-b9ac4299-b32b-45ae-9620-f2d411e07e39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:27.338467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:27.338467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:27.338467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:27.338467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:27.338467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:27.338467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:27.338467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:27.338467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:27.338467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:27.338467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:27.338467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:27.338467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:27.338467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:27.338467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:27.338467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:27.339483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:27.339483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:27.339483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:27.339483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:27.339483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:27.339483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:27.339483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:27.339483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:27.339483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:27.339483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:27.339483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:27.339483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:27.339483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:27.339483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:27.339483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:27.339483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:27.339483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:27.340372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:27.340372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:27.340372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:27.345655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 371/725] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:27.353105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac4299-b32b-45ae-9620-f2d411e07e39 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:27.435064 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 355/726] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:27.496730 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac4299-b32b-45ae-9620-f2d411e07e39 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:27.556642 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c83573-1e4e-48dd-b6db-cde225aa8d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:27.557192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c83573-1e4e-48dd-b6db-cde225aa8d0c None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.557468 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c83573-1e4e-48dd-b6db-cde225aa8d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.557742 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c83573-1e4e-48dd-b6db-cde225aa8d0c None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.558389 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c83573-1e4e-48dd-b6db-cde225aa8d0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:27.558695 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c83573-1e4e-48dd-b6db-cde225aa8d0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:27.560085 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c83573-1e4e-48dd-b6db-cde225aa8d0c None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:27.580267 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 372/727] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:27.585517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5de416-06ff-406b-a406-446334ae0ee3 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:27.636861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed329f-d02d-4f8f-aa12-c111dfaa09c4 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.637517 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed329f-d02d-4f8f-aa12-c111dfaa09c4 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.638016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed329f-d02d-4f8f-aa12-c111dfaa09c4 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.639833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 373/728] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:27.645700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b0bcb-59b3-4881-9688-0a4db84951fc None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.646241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b0bcb-59b3-4881-9688-0a4db84951fc None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.646710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b0bcb-59b3-4881-9688-0a4db84951fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.667193 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-db9b0bcb-59b3-4881-9688-0a4db84951fc None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:27.668731 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-db9b0bcb-59b3-4881-9688-0a4db84951fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:27.743513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:27.754176 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:27.768612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5de416-06ff-406b-a406-446334ae0ee3 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:27.808273 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 374/729] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 14:45:27.814680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b0bcb-59b3-4881-9688-0a4db84951fc None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:27.841014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6a308-8e54-44c4-a24e-467e7635962f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:27.841664 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6a308-8e54-44c4-a24e-467e7635962f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.841943 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6a308-8e54-44c4-a24e-467e7635962f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.842215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6a308-8e54-44c4-a24e-467e7635962f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.854152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6a308-8e54-44c4-a24e-467e7635962f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:27.855991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6a308-8e54-44c4-a24e-467e7635962f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:27.869044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-88d6a308-8e54-44c4-a24e-467e7635962f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:27.869044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:27.869044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:27.869044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:27.869044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:27.869044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:27.869044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:27.869044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:27.869044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:27.869044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:27.869044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:27.869044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:27.869044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:27.869044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:27.869044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:27.869044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:27.870008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:27.870959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:27.870959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:27.870959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:27.870959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:27.870959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:27.870959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:27.870959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:27.870959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:27.870959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:27.870959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:27.873229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 356/730] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Sep 22 14:45:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:27.874584 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7dd77-4397-4286-a3e9-a2a8cc99d867 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.875632 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7dd77-4397-4286-a3e9-a2a8cc99d867 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.876058 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7dd77-4397-4286-a3e9-a2a8cc99d867 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.877506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 357/731] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:27.884053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7dd77-4397-4286-a3e9-a2a8cc99d867 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:27.933636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b0bcb-59b3-4881-9688-0a4db84951fc None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:27.970156 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728b0f6-77f5-47e8-ba9b-9ecbcf1f0eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:27.970820 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728b0f6-77f5-47e8-ba9b-9ecbcf1f0eba None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.971100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728b0f6-77f5-47e8-ba9b-9ecbcf1f0eba None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.971387 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728b0f6-77f5-47e8-ba9b-9ecbcf1f0eba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:27.982242 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728b0f6-77f5-47e8-ba9b-9ecbcf1f0eba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:27.987254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-e728b0f6-77f5-47e8-ba9b-9ecbcf1f0eba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:27.987254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:27.987254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:27.987254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:27.987254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:27.987254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:27.987254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:27.987254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:27.987254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:27.987254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:27.987254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:27.987254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:27.987254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:27.987254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:27.987254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:27.987254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:27.988394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:27.988394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:27.988394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:27.988394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:27.988394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:27.988394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:27.988394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:27.988394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:27.988394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:27.988394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:27.988394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:27.988394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:27.988394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:27.988394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:27.988394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:27.988394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:27.989442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:27.989442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:27.989442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:27.989442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:27.989442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:27.989442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:27.989442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:27.989442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:27.989442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:27.989442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:27.989442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:27.989442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:27.989442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 375/732] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:27.990215 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d8c00-e378-4dd6-87a9-4eb27662697e None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:27.990215 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d8c00-e378-4dd6-87a9-4eb27662697e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:27.994933 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d8c00-e378-4dd6-87a9-4eb27662697e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.010792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7dd77-4397-4286-a3e9-a2a8cc99d867 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.011585 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-eb6d8c00-e378-4dd6-87a9-4eb27662697e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:28.012941 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-eb6d8c00-e378-4dd6-87a9-4eb27662697e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:28.045585 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:28.054491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9262b-d61f-43a5-86ad-af722609f89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:28.055424 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9262b-d61f-43a5-86ad-af722609f89f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.055842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9262b-d61f-43a5-86ad-af722609f89f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.056249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9262b-d61f-43a5-86ad-af722609f89f None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.057145 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9262b-d61f-43a5-86ad-af722609f89f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:28.057598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9262b-d61f-43a5-86ad-af722609f89f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:28.059255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9262b-d61f-43a5-86ad-af722609f89f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:28.079242 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 358/733] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Sep 22 14:45:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:28.080374 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9262b-d61f-43a5-86ad-af722609f89f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:28.083449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:28.101077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 376/734] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:28.101703 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d8c00-e378-4dd6-87a9-4eb27662697e None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:28.191605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9262b-d61f-43a5-86ad-af722609f89f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.218423 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d8c00-e378-4dd6-87a9-4eb27662697e None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.231410 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ade417-e6ac-43e6-9dc5-9a7403327115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:28.232026 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ade417-e6ac-43e6-9dc5-9a7403327115 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.232301 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ade417-e6ac-43e6-9dc5-9a7403327115 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.232581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ade417-e6ac-43e6-9dc5-9a7403327115 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.233245 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ade417-e6ac-43e6-9dc5-9a7403327115 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:28.233552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ade417-e6ac-43e6-9dc5-9a7403327115 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:28.234970 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ade417-e6ac-43e6-9dc5-9a7403327115 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:28.248061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 359/735] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:28.248061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ade417-e6ac-43e6-9dc5-9a7403327115 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:28.259635 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f6209-baca-4954-aef3-1e6a4030f4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:28.260348 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f6209-baca-4954-aef3-1e6a4030f4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.260632 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f6209-baca-4954-aef3-1e6a4030f4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.260903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f6209-baca-4954-aef3-1e6a4030f4a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.273252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f6209-baca-4954-aef3-1e6a4030f4a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:28.275365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f6209-baca-4954-aef3-1e6a4030f4a9 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:28.290906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-423f6209-baca-4954-aef3-1e6a4030f4a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:28.290906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:28.290906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:28.290906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:28.290906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:28.290906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:28.290906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:28.290906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:28.290906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:28.290906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:28.290906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:28.290906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:28.290906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:28.290906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:28.290906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:28.290906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:28.291879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:28.291879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:28.291879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:28.291879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:28.291879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:28.291879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:28.291879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:28.291879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:28.291879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:28.291879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:28.291879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:28.291879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:28.291879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:28.291879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:28.291879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:28.291879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:28.291879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:28.292859 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:28.292859 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:28.292859 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:28.292859 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 377/736] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:28.293926 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70918c6b-2271-4a4c-b523-accc1e96c807 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.293926 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70918c6b-2271-4a4c-b523-accc1e96c807 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.293926 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70918c6b-2271-4a4c-b523-accc1e96c807 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.298889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 378/737] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:28.298889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70918c6b-2271-4a4c-b523-accc1e96c807 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:28.321144 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:28.329561 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:28.366029 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ade417-e6ac-43e6-9dc5-9a7403327115 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.404589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade2067-5065-4454-8a25-1c13e7943fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:28.405192 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade2067-5065-4454-8a25-1c13e7943fcf None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.405463 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade2067-5065-4454-8a25-1c13e7943fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.405728 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade2067-5065-4454-8a25-1c13e7943fcf None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.407026 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade2067-5065-4454-8a25-1c13e7943fcf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:28.408378 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade2067-5065-4454-8a25-1c13e7943fcf None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:28.415158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70918c6b-2271-4a4c-b523-accc1e96c807 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.424563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 360/738] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Sep 22 14:45:28 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:28.425391 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade2067-5065-4454-8a25-1c13e7943fcf None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:28.472147 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68567e5a-8148-4251-bede-f0cd944a95ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:28.472860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68567e5a-8148-4251-bede-f0cd944a95ee None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.473141 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68567e5a-8148-4251-bede-f0cd944a95ee None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.473416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68567e5a-8148-4251-bede-f0cd944a95ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.485460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68567e5a-8148-4251-bede-f0cd944a95ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:28.487008 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68567e5a-8148-4251-bede-f0cd944a95ee None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:28.505232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-68567e5a-8148-4251-bede-f0cd944a95ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:28.505232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:28.505232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:28.505232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:28.505232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:28.505232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:28.505232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:28.505232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:28.505232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:28.505232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:28.505232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:28.505232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:28.505232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:28.505232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:28.505232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:28.505232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:28.506284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:28.506284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:28.506284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:28.506284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:28.506284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:28.506284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:28.506284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:28.506284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:28.506284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:28.506284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:28.506284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:28.506284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:28.506284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:28.506284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:28.506284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:28.506284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:28.506284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:28.507234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:28.507234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:28.507234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:28.507234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 379/739] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:28.507923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89763148-b85e-49cd-ac1b-f034c11321d1 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.508027 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89763148-b85e-49cd-ac1b-f034c11321d1 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.508293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89763148-b85e-49cd-ac1b-f034c11321d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.528515 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-89763148-b85e-49cd-ac1b-f034c11321d1 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:28.529770 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-89763148-b85e-49cd-ac1b-f034c11321d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:28.548242 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade2067-5065-4454-8a25-1c13e7943fcf None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.575106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:28.590245 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042d6c9-74e5-4e28-b74a-9a3f400b9328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:28.590997 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042d6c9-74e5-4e28-b74a-9a3f400b9328 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.591347 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042d6c9-74e5-4e28-b74a-9a3f400b9328 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.591697 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042d6c9-74e5-4e28-b74a-9a3f400b9328 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.592422 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042d6c9-74e5-4e28-b74a-9a3f400b9328 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:28.592726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042d6c9-74e5-4e28-b74a-9a3f400b9328 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:28.594172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042d6c9-74e5-4e28-b74a-9a3f400b9328 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:28.599555 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:28.608203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 361/740] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:28.609045 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042d6c9-74e5-4e28-b74a-9a3f400b9328 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:28.614883 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 380/741] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:28 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 14:45:28.615811 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89763148-b85e-49cd-ac1b-f034c11321d1 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:28.731711 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042d6c9-74e5-4e28-b74a-9a3f400b9328 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.740296 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89763148-b85e-49cd-ac1b-f034c11321d1 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.769079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b4919-e019-4ab6-820c-5bde5a9f7bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:28.769933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b4919-e019-4ab6-820c-5bde5a9f7bd2 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.770409 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b4919-e019-4ab6-820c-5bde5a9f7bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.770875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b4919-e019-4ab6-820c-5bde5a9f7bd2 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.771722 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b4919-e019-4ab6-820c-5bde5a9f7bd2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:28.773297 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b4919-e019-4ab6-820c-5bde5a9f7bd2 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:28.779006 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fde1b-a4e9-43eb-84c3-745ec36c1b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:28.779620 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fde1b-a4e9-43eb-84c3-745ec36c1b8f None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:28.779895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fde1b-a4e9-43eb-84c3-745ec36c1b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:28.780319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fde1b-a4e9-43eb-84c3-745ec36c1b8f None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:28.780934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2fde1b-a4e9-43eb-84c3-745ec36c1b8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:28.781302 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2fde1b-a4e9-43eb-84c3-745ec36c1b8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:28.782849 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2fde1b-a4e9-43eb-84c3-745ec36c1b8f None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:28.791707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 362/742] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Sep 22 14:45:28 2022] POST /identity/v3/services => generated 240 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:28.798851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 381/743] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:28.799684 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b4919-e019-4ab6-820c-5bde5a9f7bd2 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:28.803379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fde1b-a4e9-43eb-84c3-745ec36c1b8f None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:28.898133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:28.914619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:28.968500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b4919-e019-4ab6-820c-5bde5a9f7bd2 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:28.971385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fde1b-a4e9-43eb-84c3-745ec36c1b8f None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:29.020238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ebb17-71d9-42a1-b62e-80960ff28fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.020886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ebb17-71d9-42a1-b62e-80960ff28fbd None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.021159 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ebb17-71d9-42a1-b62e-80960ff28fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.021424 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ebb17-71d9-42a1-b62e-80960ff28fbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.022965 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34131a78-f6e2-4b69-9ec6-9a0a36d5502d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.023776 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34131a78-f6e2-4b69-9ec6-9a0a36d5502d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.024132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34131a78-f6e2-4b69-9ec6-9a0a36d5502d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.024561 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34131a78-f6e2-4b69-9ec6-9a0a36d5502d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.028928 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ebb17-71d9-42a1-b62e-80960ff28fbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.030393 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ebb17-71d9-42a1-b62e-80960ff28fbd None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.038020 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34131a78-f6e2-4b69-9ec6-9a0a36d5502d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.039498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34131a78-f6e2-4b69-9ec6-9a0a36d5502d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.045638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-a98ebb17-71d9-42a1-b62e-80960ff28fbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:29.045638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:29.045638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:29.045638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:29.045638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:29.045638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:29.045638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:29.045638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:29.045638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:29.045638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:29.045638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:29.045638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:29.045638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:29.045638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:29.045638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:29.046720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:29.046720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:29.046720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:29.046720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:29.046720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:29.046720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:29.046720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:29.046720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:29.046720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:29.046720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:29.046720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:29.046720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:29.046720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:29.046720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:29.046720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:29.046720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:29.046720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:29.047746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:29.047746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:29.047746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:29.047746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:29.047746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:29.047746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:29.047746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:29.047746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:29.048183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 363/744] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:29.048893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ebb17-71d9-42a1-b62e-80960ff28fbd None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:29.061910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-34131a78-f6e2-4b69-9ec6-9a0a36d5502d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:29.061910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:29.061910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:29.061910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:29.061910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:29.061910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:29.061910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:29.061910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:29.061910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:29.061910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:29.061910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:29.061910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:29.061910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:29.061910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:29.061910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:29.061910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:29.063154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:29.064321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:29.064321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:29.064321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:29.064321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:29.064321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:29.064321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:29.064321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:29.064321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:29.064321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:29.064321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:29.066604 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 382/745] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:29.067736 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34131a78-f6e2-4b69-9ec6-9a0a36d5502d None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:29.173204 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:29.176326 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ebb17-71d9-42a1-b62e-80960ff28fbd None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:29.213006 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34131a78-f6e2-4b69-9ec6-9a0a36d5502d None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:29.213832 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e3462-b474-45de-8aad-7634bf3fa274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.214587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e3462-b474-45de-8aad-7634bf3fa274 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.214968 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e3462-b474-45de-8aad-7634bf3fa274 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.215389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e3462-b474-45de-8aad-7634bf3fa274 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.226796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e3462-b474-45de-8aad-7634bf3fa274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:29.233492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-385e3462-b474-45de-8aad-7634bf3fa274 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:29.233492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:29.233492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:29.233492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:29.233492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:29.233492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:29.233492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:29.233492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:29.233492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:29.233492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:29.233492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:29.233492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:29.233492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:29.233492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:29.233492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:29.233492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:29.235240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:29.235240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:29.235240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:29.235240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:29.235240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:29.235240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:29.235240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:29.235240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:29.235240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:29.235240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:29.235240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:29.235240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:29.235240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:29.235240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:29.235240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:29.235240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:29.236263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:29.236263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:29.236263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:29.236263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:29.236263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:29.236263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:29.236263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:29.236263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:29.236263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:29.236263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:29.236263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:29.236263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:29.236993 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 364/746] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:29.237570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:29.237893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e3462-b474-45de-8aad-7634bf3fa274 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:29.263340 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5703c-8881-46d2-b531-3fcebe63fb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.264536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5703c-8881-46d2-b531-3fcebe63fb9a None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.264967 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5703c-8881-46d2-b531-3fcebe63fb9a None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.265389 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5703c-8881-46d2-b531-3fcebe63fb9a None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.267315 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5703c-8881-46d2-b531-3fcebe63fb9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:29.268104 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5703c-8881-46d2-b531-3fcebe63fb9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.269928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5703c-8881-46d2-b531-3fcebe63fb9a None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.287219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 383/747] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:29.288694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5703c-8881-46d2-b531-3fcebe63fb9a None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:29.360040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e3462-b474-45de-8aad-7634bf3fa274 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:29.399769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1e0a3-80b6-4369-a04a-7f480a09289a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.400858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1e0a3-80b6-4369-a04a-7f480a09289a None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.401143 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1e0a3-80b6-4369-a04a-7f480a09289a None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.401415 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1e0a3-80b6-4369-a04a-7f480a09289a None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.402138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1e0a3-80b6-4369-a04a-7f480a09289a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:29.402442 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1e0a3-80b6-4369-a04a-7f480a09289a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.404652 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1e0a3-80b6-4369-a04a-7f480a09289a None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.434189 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 365/748] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Sep 22 14:45:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:29.434189 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1e0a3-80b6-4369-a04a-7f480a09289a None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:29.435489 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5703c-8881-46d2-b531-3fcebe63fb9a None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:29.456735 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:29.485406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59516239-dbe0-4aaf-a0fc-e6bd26eac916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.486266 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59516239-dbe0-4aaf-a0fc-e6bd26eac916 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.486694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59516239-dbe0-4aaf-a0fc-e6bd26eac916 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.487169 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59516239-dbe0-4aaf-a0fc-e6bd26eac916 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.488482 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59516239-dbe0-4aaf-a0fc-e6bd26eac916 None admin] RBAC: Adding query filter params (name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:29.488936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59516239-dbe0-4aaf-a0fc-e6bd26eac916 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.491098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59516239-dbe0-4aaf-a0fc-e6bd26eac916 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.506723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 384/749] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:29.506723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59516239-dbe0-4aaf-a0fc-e6bd26eac916 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:29.558617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:29.571081 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1e0a3-80b6-4369-a04a-7f480a09289a None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:29.613097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd880e9d-30bb-4ffc-bf5f-1384030deb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.614396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd880e9d-30bb-4ffc-bf5f-1384030deb86 None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.615124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd880e9d-30bb-4ffc-bf5f-1384030deb86 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.615789 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd880e9d-30bb-4ffc-bf5f-1384030deb86 None admin] PATH_INFO: `/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/0c9be41c39494e54a14398a1f03c4d69/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.619019 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd880e9d-30bb-4ffc-bf5f-1384030deb86 None admin] RBAC: Authorizing `identity:create_grant(project_id=43bc5871e4bf441b874474bcae4e5385, user_id=0c9be41c39494e54a14398a1f03c4d69, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.620604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd880e9d-30bb-4ffc-bf5f-1384030deb86 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.639244 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59516239-dbe0-4aaf-a0fc-e6bd26eac916 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:29.647032 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 366/750] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:29 2022] PUT /identity/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/0c9be41c39494e54a14398a1f03c4d69/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:45:29.648297 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd880e9d-30bb-4ffc-bf5f-1384030deb86 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:29.684899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee95ad-1e7a-4e4d-8d99-81cc826c7c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.685572 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee95ad-1e7a-4e4d-8d99-81cc826c7c92 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.685940 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee95ad-1e7a-4e4d-8d99-81cc826c7c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.686346 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee95ad-1e7a-4e4d-8d99-81cc826c7c92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.774745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee95ad-1e7a-4e4d-8d99-81cc826c7c92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.776470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee95ad-1e7a-4e4d-8d99-81cc826c7c92 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.821205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:29.828772 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:29.890977 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd880e9d-30bb-4ffc-bf5f-1384030deb86 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:29.902501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 385/751] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:45:29.911934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee95ad-1e7a-4e4d-8d99-81cc826c7c92 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:29.940621 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83c2a2-a027-475e-9432-026af9ae433d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:29.941184 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83c2a2-a027-475e-9432-026af9ae433d None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:29.941461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83c2a2-a027-475e-9432-026af9ae433d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:29.941723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83c2a2-a027-475e-9432-026af9ae433d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:29.957542 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83c2a2-a027-475e-9432-026af9ae433d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:29.959302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83c2a2-a027-475e-9432-026af9ae433d None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:29.980691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-fe83c2a2-a027-475e-9432-026af9ae433d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:29.980691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:29.980691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:29.980691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:29.980691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:29.980691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:29.980691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:29.980691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:29.980691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:29.980691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:29.980691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:29.980691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:29.980691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:29.980691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:29.980691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:29.980691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:29.981714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:29.981714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:29.981714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:29.981714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:29.981714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:29.981714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:29.981714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:29.981714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:29.981714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:29.981714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:29.981714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:29.981714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:29.981714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:29.981714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:29.981714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:29.981714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:29.981714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:29.982639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:29.982639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:29.982639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:29.982639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 367/752] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:29.989628 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83c2a2-a027-475e-9432-026af9ae433d None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:30.061152 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee95ad-1e7a-4e4d-8d99-81cc826c7c92 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:30.102090 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26613c01-c543-4e3e-82e7-9a7310734871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:30.102898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26613c01-c543-4e3e-82e7-9a7310734871 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.103178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26613c01-c543-4e3e-82e7-9a7310734871 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.103446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26613c01-c543-4e3e-82e7-9a7310734871 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.111481 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:30.117179 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26613c01-c543-4e3e-82e7-9a7310734871 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:30.119000 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26613c01-c543-4e3e-82e7-9a7310734871 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:30.136833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-26613c01-c543-4e3e-82e7-9a7310734871 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:30.136833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:30.136833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:30.136833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:30.136833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:30.136833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:30.136833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:30.136833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:30.136833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:30.136833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:30.136833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:30.136833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:30.136833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:30.136833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:30.136833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:30.136833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:30.137991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:30.137991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:30.137991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:30.137991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:30.137991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:30.137991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:30.137991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:30.137991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:30.137991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:30.137991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:30.137991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:30.137991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:30.137991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:30.137991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:30.137991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:30.137991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:30.137991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:30.138910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:30.138910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:30.138910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:30.141981 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 386/753] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:30.150364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26613c01-c543-4e3e-82e7-9a7310734871 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:30.264407 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26613c01-c543-4e3e-82e7-9a7310734871 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:30.275930 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83c2a2-a027-475e-9432-026af9ae433d None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:30.305146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397d552-b190-4d90-b82d-dec20acbf400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:30.306129 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397d552-b190-4d90-b82d-dec20acbf400 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.306787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397d552-b190-4d90-b82d-dec20acbf400 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.307284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397d552-b190-4d90-b82d-dec20acbf400 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.308184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397d552-b190-4d90-b82d-dec20acbf400 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:30.308726 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397d552-b190-4d90-b82d-dec20acbf400 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:30.310343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397d552-b190-4d90-b82d-dec20acbf400 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:30.326178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 387/754] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:30.335860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397d552-b190-4d90-b82d-dec20acbf400 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:30.355088 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:30.363905 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647fcb8-6936-4a2a-bdc6-7d02e3023f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:30.364534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647fcb8-6936-4a2a-bdc6-7d02e3023f7c None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.364814 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647fcb8-6936-4a2a-bdc6-7d02e3023f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.365093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647fcb8-6936-4a2a-bdc6-7d02e3023f7c None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.365746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7647fcb8-6936-4a2a-bdc6-7d02e3023f7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:30.366052 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7647fcb8-6936-4a2a-bdc6-7d02e3023f7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:30.368289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7647fcb8-6936-4a2a-bdc6-7d02e3023f7c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:30.390385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 368/755] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:30.397083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647fcb8-6936-4a2a-bdc6-7d02e3023f7c None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:30.448650 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:30.504505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397d552-b190-4d90-b82d-dec20acbf400 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:30.542245 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d137e36-07d6-4baf-9f4f-71008da68b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:30.542939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d137e36-07d6-4baf-9f4f-71008da68b2f None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.543265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d137e36-07d6-4baf-9f4f-71008da68b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.543542 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d137e36-07d6-4baf-9f4f-71008da68b2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.551449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d137e36-07d6-4baf-9f4f-71008da68b2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:30.553224 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d137e36-07d6-4baf-9f4f-71008da68b2f None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:30.558556 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647fcb8-6936-4a2a-bdc6-7d02e3023f7c None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:30.566295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-5d137e36-07d6-4baf-9f4f-71008da68b2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:30.566295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:30.566295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:30.566295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:30.566295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:30.566295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:30.566295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:30.566295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:30.566295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:30.566295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:30.566295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:30.566295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:30.566295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:30.566295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:30.566295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:30.567310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:30.567310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:30.567310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:30.567310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:30.567310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:30.567310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:30.567310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:30.567310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:30.567310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:30.567310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:30.567310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:30.567310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:30.567310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:30.567310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:30.567310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:30.567310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:30.567310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:30.568219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:30.568219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:30.568219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:30.568219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:30.568219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:30.568219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:30.568219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:30.568219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:30.568219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 388/756] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:30.574685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d137e36-07d6-4baf-9f4f-71008da68b2f None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:30.616916 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2d235-bc42-48cc-b33b-d765c80e6a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:30.617579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2d235-bc42-48cc-b33b-d765c80e6a5a None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.617859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2d235-bc42-48cc-b33b-d765c80e6a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.618130 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2d235-bc42-48cc-b33b-d765c80e6a5a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.630969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e2d235-bc42-48cc-b33b-d765c80e6a5a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:30.632445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e2d235-bc42-48cc-b33b-d765c80e6a5a None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:30.667177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-18e2d235-bc42-48cc-b33b-d765c80e6a5a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:30.667177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:30.667177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:30.667177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:30.667177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:30.667177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:30.667177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:30.667177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:30.667177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:30.667177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:30.667177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:30.667177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:30.667177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:30.667177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:30.667177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:30.667177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:30.669371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:30.671572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:30.671572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:30.671572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:30.671572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:30.671572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:30.671572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:30.671572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:30.671572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:30.671572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:30.671572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:30.671572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 369/757] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Sep 22 14:45:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:30.678907 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2d235-bc42-48cc-b33b-d765c80e6a5a None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:30.688238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d137e36-07d6-4baf-9f4f-71008da68b2f None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:30.699407 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:30.726518 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c9828-c52c-4cd4-a762-34ecf1bfbea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:30.727164 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c9828-c52c-4cd4-a762-34ecf1bfbea7 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.727440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c9828-c52c-4cd4-a762-34ecf1bfbea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.727768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c9828-c52c-4cd4-a762-34ecf1bfbea7 None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.728469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c9828-c52c-4cd4-a762-34ecf1bfbea7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:30.728774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c9828-c52c-4cd4-a762-34ecf1bfbea7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:30.730288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c9828-c52c-4cd4-a762-34ecf1bfbea7 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:30.745042 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 389/758] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:30.755589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c9828-c52c-4cd4-a762-34ecf1bfbea7 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:30.775629 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:30.824542 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2d235-bc42-48cc-b33b-d765c80e6a5a None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:30.861555 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daef1e0-bd6b-4402-8379-42418fb64b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:30.862340 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daef1e0-bd6b-4402-8379-42418fb64b82 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.862723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daef1e0-bd6b-4402-8379-42418fb64b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.863123 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daef1e0-bd6b-4402-8379-42418fb64b82 None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.863956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daef1e0-bd6b-4402-8379-42418fb64b82 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:30.864366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daef1e0-bd6b-4402-8379-42418fb64b82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:30.865908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daef1e0-bd6b-4402-8379-42418fb64b82 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:30.885449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 370/759] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Sep 22 14:45:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:30.886916 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c87ecd-a1f1-4668-bc8c-24558da5fb56 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.887305 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c87ecd-a1f1-4668-bc8c-24558da5fb56 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.887681 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c87ecd-a1f1-4668-bc8c-24558da5fb56 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.889293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 371/760] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:30.890595 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c87ecd-a1f1-4668-bc8c-24558da5fb56 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:30.912292 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c9828-c52c-4cd4-a762-34ecf1bfbea7 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:30.948757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cff93c-b842-46e3-b25a-7bcf0c442f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:30.949523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cff93c-b842-46e3-b25a-7bcf0c442f66 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.949806 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cff93c-b842-46e3-b25a-7bcf0c442f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.950077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cff93c-b842-46e3-b25a-7bcf0c442f66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.950943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cff93c-b842-46e3-b25a-7bcf0c442f66 None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=0a265623ae734936a340c97df852e227) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:30.951252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cff93c-b842-46e3-b25a-7bcf0c442f66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:30.952768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cff93c-b842-46e3-b25a-7bcf0c442f66 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:30.959825 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 390/761] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:30 2022] GET /identity/v3/role_assignments?user.id=0a265623ae734936a340c97df852e227&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=32c8a81e339a462e973e76bb528ef35a => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:30.961155 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e4f42-7c3c-46b9-b81a-7eccd51a2c9b None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:30.961487 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e4f42-7c3c-46b9-b81a-7eccd51a2c9b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:30.961765 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e4f42-7c3c-46b9-b81a-7eccd51a2c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:30.981207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-cd6e4f42-7c3c-46b9-b81a-7eccd51a2c9b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:30.982513 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-cd6e4f42-7c3c-46b9-b81a-7eccd51a2c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:31.016267 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:31.044715 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c87ecd-a1f1-4668-bc8c-24558da5fb56 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:31.059421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 391/762] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:31.075208 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:31.113801 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024d643-8149-4100-a03b-38351cbf27bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:31.114455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024d643-8149-4100-a03b-38351cbf27bd None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:31.114751 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024d643-8149-4100-a03b-38351cbf27bd None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:31.115030 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024d643-8149-4100-a03b-38351cbf27bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:31.207442 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024d643-8149-4100-a03b-38351cbf27bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:31.208943 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024d643-8149-4100-a03b-38351cbf27bd None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:31.254146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b02b23-46fe-4a53-ad99-1ae1b0d28210 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:31.254500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b02b23-46fe-4a53-ad99-1ae1b0d28210 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:31.254700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b02b23-46fe-4a53-ad99-1ae1b0d28210 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:31.256457 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 392/763] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:31.263312 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c11773-c4e0-4afd-a11a-605f4eeff6db None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:31.263619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c11773-c4e0-4afd-a11a-605f4eeff6db None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:31.263864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c11773-c4e0-4afd-a11a-605f4eeff6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:31.298858 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-68c11773-c4e0-4afd-a11a-605f4eeff6db None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:31.300074 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-68c11773-c4e0-4afd-a11a-605f4eeff6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:31.320773 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 372/764] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:45:31.332660 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024d643-8149-4100-a03b-38351cbf27bd None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:31.347185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:31.430005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 393/765] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:31.431438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c11773-c4e0-4afd-a11a-605f4eeff6db None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:31.480344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024d643-8149-4100-a03b-38351cbf27bd None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:31.518642 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4382dd8-7694-4a58-bdf6-e146a4730e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:31.519317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4382dd8-7694-4a58-bdf6-e146a4730e40 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:31.519625 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4382dd8-7694-4a58-bdf6-e146a4730e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:31.520342 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4382dd8-7694-4a58-bdf6-e146a4730e40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:31.533310 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4382dd8-7694-4a58-bdf6-e146a4730e40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:31.534950 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4382dd8-7694-4a58-bdf6-e146a4730e40 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:31.542814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:31.553533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-c4382dd8-7694-4a58-bdf6-e146a4730e40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:31.553533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:31.553533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:31.553533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:31.553533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:31.553533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:31.553533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:31.553533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:31.553533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:31.553533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:31.553533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:31.553533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:31.553533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:31.553533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:31.553533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:31.553533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:31.554541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:31.554541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:31.554541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:31.554541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:31.554541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:31.554541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:31.554541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:31.554541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:31.554541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:31.554541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:31.554541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:31.554541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:31.554541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:31.554541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:31.554541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:31.554541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:31.554541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:31.555497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:31.555497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:31.555497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:31.556622 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c11773-c4e0-4afd-a11a-605f4eeff6db None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:31.557966 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 373/766] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:31.563094 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4382dd8-7694-4a58-bdf6-e146a4730e40 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:31.711242 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:31.758347 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4382dd8-7694-4a58-bdf6-e146a4730e40 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:31.768714 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab809e9-a24a-4369-9726-e233fc8fba85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:31.769381 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab809e9-a24a-4369-9726-e233fc8fba85 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:31.769662 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab809e9-a24a-4369-9726-e233fc8fba85 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:31.769935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab809e9-a24a-4369-9726-e233fc8fba85 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:31.770525 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab809e9-a24a-4369-9726-e233fc8fba85 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:31.772525 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab809e9-a24a-4369-9726-e233fc8fba85 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:31.782813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-bab809e9-a24a-4369-9726-e233fc8fba85 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:31.782813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:31.782813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:31.782813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:31.782813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:31.782813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:31.782813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:31.782813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:31.782813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:31.782813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:31.782813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:31.782813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:31.782813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:31.782813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:31.782813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:31.782813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:31.782813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:31.783831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:31.783831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:31.783831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:31.783831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:31.783831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:31.783831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:31.783831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:31.783831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:31.783831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:31.783831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:31.783831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:31.783831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:31.783831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:31.783831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:31.783831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:31.783831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:31.784748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:31.784748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:31.784748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:31.784748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:31.784748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:31.784748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:31.784748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:31.784748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:31.784748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:31.784748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:31.784748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:31.784748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:31.784748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:31.784748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:31.784748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 394/767] 10.222.0.26 () {56 vars in 1129 bytes} [Thu Sep 22 14:45:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:31.785570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab809e9-a24a-4369-9726-e233fc8fba85 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:31.799647 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cb70c-c5d7-4b0f-b1f2-7b03433ed953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:31.800539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cb70c-c5d7-4b0f-b1f2-7b03433ed953 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:31.801028 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cb70c-c5d7-4b0f-b1f2-7b03433ed953 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:31.801507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cb70c-c5d7-4b0f-b1f2-7b03433ed953 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:31.802227 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59cb70c-c5d7-4b0f-b1f2-7b03433ed953 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:31.802620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59cb70c-c5d7-4b0f-b1f2-7b03433ed953 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:31.804068 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59cb70c-c5d7-4b0f-b1f2-7b03433ed953 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:31.817309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 374/768] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:31.822495 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cb70c-c5d7-4b0f-b1f2-7b03433ed953 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:31.913051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab809e9-a24a-4369-9726-e233fc8fba85 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:31.934317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cb70c-c5d7-4b0f-b1f2-7b03433ed953 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:31.955547 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8ef75-11d3-4829-b715-d54c49649d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:31.956329 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8ef75-11d3-4829-b715-d54c49649d52 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:31.956719 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8ef75-11d3-4829-b715-d54c49649d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:31.957075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c8ef75-11d3-4829-b715-d54c49649d52 None admin] PATH_INFO: `/v3/services/network` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:31.957752 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8ef75-11d3-4829-b715-d54c49649d52 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:31.959296 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c8ef75-11d3-4829-b715-d54c49649d52 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:31.961829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:31.970355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-a6c8ef75-11d3-4829-b715-d54c49649d52 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:31.970355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:31.970355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:31.970355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:31.970355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:31.970355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:31.970355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:31.970355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:31.970355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:31.970355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:31.970355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:31.970355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:31.970355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:31.970355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:31.970355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:31.970355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:31.970355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:31.971869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:31.971869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:31.971869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:31.971869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:31.971869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:31.971869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:31.971869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:31.971869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:31.971869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:31.971869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:31.971869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:31.971869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:31.971869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:31.971869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:31.971869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:31.971869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:31.972910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:31.972910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:31.972910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:31.972910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:31.972910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:31.972910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:31.972910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:31.972910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:31.972910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:31.972910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:31.972910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:31.972910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:31.972910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:31.972910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:31.979249 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 395/769] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:31 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:31.980256 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8ef75-11d3-4829-b715-d54c49649d52 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:31.982899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf91482-3a60-4fa3-9f5b-ccfcea05040c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:31.983887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf91482-3a60-4fa3-9f5b-ccfcea05040c None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:31.984836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf91482-3a60-4fa3-9f5b-ccfcea05040c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:31.986078 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf91482-3a60-4fa3-9f5b-ccfcea05040c None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:31.986889 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf91482-3a60-4fa3-9f5b-ccfcea05040c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:31.987359 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf91482-3a60-4fa3-9f5b-ccfcea05040c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:31.989013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf91482-3a60-4fa3-9f5b-ccfcea05040c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:31.996841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 375/770] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Sep 22 14:45:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:31.998167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf91482-3a60-4fa3-9f5b-ccfcea05040c None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:32.009270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:32.115525 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c8ef75-11d3-4829-b715-d54c49649d52 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:32.119736 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf91482-3a60-4fa3-9f5b-ccfcea05040c None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:32.157769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b564fc4-170b-4fdb-a152-4bd29bb07cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:32.158532 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b564fc4-170b-4fdb-a152-4bd29bb07cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.158933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b564fc4-170b-4fdb-a152-4bd29bb07cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.159285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b564fc4-170b-4fdb-a152-4bd29bb07cc2 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.160006 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b564fc4-170b-4fdb-a152-4bd29bb07cc2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:32.160510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b564fc4-170b-4fdb-a152-4bd29bb07cc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:32.162172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b564fc4-170b-4fdb-a152-4bd29bb07cc2 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:32.167982 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d6007-8921-43fa-a2db-2af3186634c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:32.168888 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d6007-8921-43fa-a2db-2af3186634c8 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.169310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d6007-8921-43fa-a2db-2af3186634c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.169709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d6007-8921-43fa-a2db-2af3186634c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.170798 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 376/771] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:32.171754 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b564fc4-170b-4fdb-a152-4bd29bb07cc2 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:32.181358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d6007-8921-43fa-a2db-2af3186634c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:32.183570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d6007-8921-43fa-a2db-2af3186634c8 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:32.202301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-e43d6007-8921-43fa-a2db-2af3186634c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:32.202301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:32.202301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:32.202301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:32.202301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:32.202301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:32.202301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:32.202301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:32.202301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:32.202301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:32.202301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:32.202301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:32.202301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:32.202301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:32.202301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:32.203459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:32.203459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:32.203459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:32.203459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:32.203459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:32.203459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:32.203459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:32.203459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:32.203459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:32.203459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:32.203459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:32.203459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:32.203459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:32.203459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:32.203459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:32.203459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:32.203459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:32.204551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:32.204551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:32.204551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:32.204551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:32.204551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:32.204551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:32.204551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:32.204551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:32.207450 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 396/772] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:32.208755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d6007-8921-43fa-a2db-2af3186634c8 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:32.232636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:32.291906 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b564fc4-170b-4fdb-a152-4bd29bb07cc2 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:32.324684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:32.342277 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2c836-a576-4c74-bb8b-736a359858d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:32.342936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2c836-a576-4c74-bb8b-736a359858d8 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.343177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2c836-a576-4c74-bb8b-736a359858d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.343484 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2c836-a576-4c74-bb8b-736a359858d8 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.345094 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2c836-a576-4c74-bb8b-736a359858d8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:32.345401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2c836-a576-4c74-bb8b-736a359858d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:32.347593 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2c836-a576-4c74-bb8b-736a359858d8 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:32.354402 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 377/773] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Sep 22 14:45:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:32.355515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2c836-a576-4c74-bb8b-736a359858d8 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:32.363997 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d6007-8921-43fa-a2db-2af3186634c8 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:32.410414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f91b2-6d95-4cbe-b45a-97d08cd57b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:32.411823 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f91b2-6d95-4cbe-b45a-97d08cd57b0f None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.412317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f91b2-6d95-4cbe-b45a-97d08cd57b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.412807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f91b2-6d95-4cbe-b45a-97d08cd57b0f None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.413657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f91b2-6d95-4cbe-b45a-97d08cd57b0f None admin] RBAC: Adding query filter params (type=network) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:32.414185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f91b2-6d95-4cbe-b45a-97d08cd57b0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:32.416863 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f91b2-6d95-4cbe-b45a-97d08cd57b0f None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:32.429076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 397/774] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:32.430352 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f91b2-6d95-4cbe-b45a-97d08cd57b0f None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:32.474794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2c836-a576-4c74-bb8b-736a359858d8 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:32.493121 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:32.521044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2973b-44a8-4424-a949-264c75ec5d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:32.521669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2973b-44a8-4424-a949-264c75ec5d44 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.521942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2973b-44a8-4424-a949-264c75ec5d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.522254 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2973b-44a8-4424-a949-264c75ec5d44 None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.522986 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2973b-44a8-4424-a949-264c75ec5d44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:32.523287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2973b-44a8-4424-a949-264c75ec5d44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:32.524733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2973b-44a8-4424-a949-264c75ec5d44 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:32.538560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 378/775] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:32.539356 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2973b-44a8-4424-a949-264c75ec5d44 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:32.568109 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f91b2-6d95-4cbe-b45a-97d08cd57b0f None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:32.605113 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba6aa9-56ef-4109-954c-33be25bb017c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:32.605725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba6aa9-56ef-4109-954c-33be25bb017c None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.607320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba6aa9-56ef-4109-954c-33be25bb017c None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.607902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba6aa9-56ef-4109-954c-33be25bb017c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.609158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba6aa9-56ef-4109-954c-33be25bb017c None admin] RBAC: Adding query filter params (interface=public, service_id=4998d014acea48c6b3434764ceb11adf, region_id=RegionOne) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:32.609682 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba6aa9-56ef-4109-954c-33be25bb017c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:32.611820 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ba6aa9-56ef-4109-954c-33be25bb017c None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:32.622001 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 398/776] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Sep 22 14:45:32 2022] GET /identity/v3/endpoints?service_id=4998d014acea48c6b3434764ceb11adf&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:32.622903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba6aa9-56ef-4109-954c-33be25bb017c None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:32.634957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:32.666177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2973b-44a8-4424-a949-264c75ec5d44 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:32.705725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b57e55-3217-472b-aa1a-eaa99ed8066e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:32.706342 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b57e55-3217-472b-aa1a-eaa99ed8066e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.706619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b57e55-3217-472b-aa1a-eaa99ed8066e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.706943 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b57e55-3217-472b-aa1a-eaa99ed8066e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.707721 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b57e55-3217-472b-aa1a-eaa99ed8066e None admin] RBAC: Adding query filter params (interface=public, service_id=0da542d0e67e47da8cb7adc58ced351c, region_id=RegionOne) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:32.707949 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b57e55-3217-472b-aa1a-eaa99ed8066e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:32.709347 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b57e55-3217-472b-aa1a-eaa99ed8066e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:32.717400 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 379/777] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Sep 22 14:45:32 2022] GET /identity/v3/endpoints?service_id=0da542d0e67e47da8cb7adc58ced351c&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:32.719272 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75be4a9-a748-46b8-bf6a-172bd0f7cfc0 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.719550 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75be4a9-a748-46b8-bf6a-172bd0f7cfc0 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.719821 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75be4a9-a748-46b8-bf6a-172bd0f7cfc0 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.725318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 380/778] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:32.725318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75be4a9-a748-46b8-bf6a-172bd0f7cfc0 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:32.748315 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba6aa9-56ef-4109-954c-33be25bb017c None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:32.759784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:32.787385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90798da-26a9-4875-9d59-acf682730be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:32.788233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90798da-26a9-4875-9d59-acf682730be4 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.788762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90798da-26a9-4875-9d59-acf682730be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.789225 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90798da-26a9-4875-9d59-acf682730be4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.790002 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90798da-26a9-4875-9d59-acf682730be4 None admin] RBAC: Adding query filter params (role.id=35bdd5122aae45ebbe44158a433d2f99, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=6256407d9df648d5bbe33495cb9a8b36) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:32.790310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90798da-26a9-4875-9d59-acf682730be4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:32.792228 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90798da-26a9-4875-9d59-acf682730be4 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:32.806765 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 399/779] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:32 2022] GET /identity/v3/role_assignments?user.id=6256407d9df648d5bbe33495cb9a8b36&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=35bdd5122aae45ebbe44158a433d2f99 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:32.806765 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90798da-26a9-4875-9d59-acf682730be4 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:32.856977 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75be4a9-a748-46b8-bf6a-172bd0f7cfc0 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:32.897259 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:32.900981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f761c0-cb2f-4413-b943-10c2b5843262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:32.902040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f761c0-cb2f-4413-b943-10c2b5843262 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.902745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f761c0-cb2f-4413-b943-10c2b5843262 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.903041 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f761c0-cb2f-4413-b943-10c2b5843262 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.903663 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f761c0-cb2f-4413-b943-10c2b5843262 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:32.905416 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f761c0-cb2f-4413-b943-10c2b5843262 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:32.912570 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 381/780] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Sep 22 14:45:32 2022] GET /identity/v3/services => generated 772 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:32.913958 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec588611-7854-4184-8e53-054c3494b972 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.914238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec588611-7854-4184-8e53-054c3494b972 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.914505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec588611-7854-4184-8e53-054c3494b972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.934894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-ec588611-7854-4184-8e53-054c3494b972 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:32.936231 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-ec588611-7854-4184-8e53-054c3494b972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:32.946880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90798da-26a9-4875-9d59-acf682730be4 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:32.992489 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87174d78-c433-4426-88c0-157ce1f794bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:32.993371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87174d78-c433-4426-88c0-157ce1f794bb None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:32.993753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87174d78-c433-4426-88c0-157ce1f794bb None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:32.994209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87174d78-c433-4426-88c0-157ce1f794bb None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:32.995110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87174d78-c433-4426-88c0-157ce1f794bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:32.996950 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87174d78-c433-4426-88c0-157ce1f794bb None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:33.008320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 400/781] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Sep 22 14:45:32 2022] GET /identity/v3/services => generated 772 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:33.057931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 382/782] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:33.343208 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87174d78-c433-4426-88c0-157ce1f794bb None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:33.440266 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ac20b-06ef-4592-b85c-3c620b6c7cd8 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:33.440777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ac20b-06ef-4592-b85c-3c620b6c7cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:33.440777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ac20b-06ef-4592-b85c-3c620b6c7cd8 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:33.442367 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 383/783] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:33.447488 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cf5f0-2c8b-499b-8f17-9abbbc0780ea None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:33.447780 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cf5f0-2c8b-499b-8f17-9abbbc0780ea None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:33.448061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cf5f0-2c8b-499b-8f17-9abbbc0780ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:33.450020 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87174d78-c433-4426-88c0-157ce1f794bb None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:33.486783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-f57cf5f0-2c8b-499b-8f17-9abbbc0780ea None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:33.488011 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f57cf5f0-2c8b-499b-8f17-9abbbc0780ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:33.516039 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884570a1-9d75-414d-b9e6-1f752da0761b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:33.516701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884570a1-9d75-414d-b9e6-1f752da0761b None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:33.516982 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884570a1-9d75-414d-b9e6-1f752da0761b None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:33.517256 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884570a1-9d75-414d-b9e6-1f752da0761b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:33.530782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:33.539234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:33.539427 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884570a1-9d75-414d-b9e6-1f752da0761b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:33.545440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-884570a1-9d75-414d-b9e6-1f752da0761b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:33.545440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:33.545440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:33.545440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:33.545440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:33.545440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:33.545440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:33.545440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:33.545440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:33.545440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:33.545440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:33.545440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:33.545440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:33.545440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:33.545440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:33.545440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:33.546430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:33.546430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:33.546430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:33.546430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:33.546430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:33.546430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:33.546430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:33.546430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:33.546430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:33.546430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:33.546430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:33.546430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:33.546430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:33.546430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:33.546430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:33.546430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:33.547899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:33.547899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:33.547899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:33.547899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:33.547899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:33.547899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:33.547899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:33.547899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:33.547899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:33.547899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:33.547899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:33.547899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:33.550094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 401/784] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:33.555485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884570a1-9d75-414d-b9e6-1f752da0761b None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:33.624217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 384/785] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:33.714859 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884570a1-9d75-414d-b9e6-1f752da0761b None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:33.760490 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b152e-39d8-4213-9f56-847d8ad5fd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:33.761487 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b152e-39d8-4213-9f56-847d8ad5fd6d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:33.761991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b152e-39d8-4213-9f56-847d8ad5fd6d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:33.762485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b152e-39d8-4213-9f56-847d8ad5fd6d None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:33.763389 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b152e-39d8-4213-9f56-847d8ad5fd6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:33.763892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b152e-39d8-4213-9f56-847d8ad5fd6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:33.765550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b152e-39d8-4213-9f56-847d8ad5fd6d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:33.778717 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 402/786] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:33.785341 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cf5f0-2c8b-499b-8f17-9abbbc0780ea None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:33.923368 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cf5f0-2c8b-499b-8f17-9abbbc0780ea None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:33.974165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea70306-105e-49db-9b56-63aa1e161059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:33.975273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea70306-105e-49db-9b56-63aa1e161059 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:33.975803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea70306-105e-49db-9b56-63aa1e161059 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:33.976278 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea70306-105e-49db-9b56-63aa1e161059 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:33.988029 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea70306-105e-49db-9b56-63aa1e161059 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:33.989748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea70306-105e-49db-9b56-63aa1e161059 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:33.999342 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b152e-39d8-4213-9f56-847d8ad5fd6d None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:34.010833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-cea70306-105e-49db-9b56-63aa1e161059 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:34.010833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:34.010833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:34.010833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:34.010833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:34.010833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:34.010833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:34.010833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:34.010833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:34.010833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:34.010833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:34.010833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:34.010833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:34.010833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:34.010833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:34.010833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:34.011853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:34.011853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:34.011853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:34.011853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:34.011853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:34.011853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:34.011853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:34.011853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:34.011853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:34.011853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:34.011853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:34.011853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:34.011853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:34.011853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:34.011853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:34.011853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:34.011853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:34.012762 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:34.012762 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:34.012762 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:34.015065 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 385/787] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:34.022862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea70306-105e-49db-9b56-63aa1e161059 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:34.054836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:34.121004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:34.176134 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b152e-39d8-4213-9f56-847d8ad5fd6d None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:34.220237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea70306-105e-49db-9b56-63aa1e161059 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:34.250606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53eea9-05ec-4c97-b7bb-9f9bee702ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:34.251256 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53eea9-05ec-4c97-b7bb-9f9bee702ffb None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:34.251535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53eea9-05ec-4c97-b7bb-9f9bee702ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:34.251802 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53eea9-05ec-4c97-b7bb-9f9bee702ffb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:34.272722 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53eea9-05ec-4c97-b7bb-9f9bee702ffb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:34.277600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-be53eea9-05ec-4c97-b7bb-9f9bee702ffb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:34.277600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:34.277600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:34.277600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:34.277600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:34.277600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:34.277600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:34.277600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:34.277600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:34.277600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:34.277600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:34.277600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:34.277600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:34.277600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:34.277600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:34.277600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:34.278725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:34.278725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:34.278725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:34.278725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:34.278725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:34.278725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:34.278725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:34.278725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:34.278725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:34.278725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:34.278725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:34.278725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:34.278725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:34.278725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:34.278725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:34.278725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:34.279813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:34.279813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:34.279813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:34.279813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:34.279813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:34.279813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:34.279813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:34.279813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:34.279813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:34.279813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:34.279813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:34.279813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:34.286764 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 403/788] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:34.288709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53eea9-05ec-4c97-b7bb-9f9bee702ffb None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:34.298756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f983de-3e45-4b35-839f-8eb7622285cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:34.298756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f983de-3e45-4b35-839f-8eb7622285cd None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:34.298756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f983de-3e45-4b35-839f-8eb7622285cd None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:34.298756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f983de-3e45-4b35-839f-8eb7622285cd None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:34.299152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f983de-3e45-4b35-839f-8eb7622285cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:34.299152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f983de-3e45-4b35-839f-8eb7622285cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:34.310197 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f983de-3e45-4b35-839f-8eb7622285cd None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:34.333209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 386/789] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:34.345043 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f983de-3e45-4b35-839f-8eb7622285cd None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:34.459353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53eea9-05ec-4c97-b7bb-9f9bee702ffb None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:34.483365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:34.488235 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:34.497766 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b1d11-3406-464c-b959-01f260d9654b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:34.498378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b1d11-3406-464c-b959-01f260d9654b None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:34.498657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b1d11-3406-464c-b959-01f260d9654b None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:34.498951 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b1d11-3406-464c-b959-01f260d9654b None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:34.499713 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b1d11-3406-464c-b959-01f260d9654b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:34.500588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b1d11-3406-464c-b959-01f260d9654b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:34.502012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b1d11-3406-464c-b959-01f260d9654b None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:34.511030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 404/790] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:34.518834 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b1d11-3406-464c-b959-01f260d9654b None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:34.563189 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f983de-3e45-4b35-839f-8eb7622285cd None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:34.607002 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6486128-5b78-4a66-afef-d4cc56a8afaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:34.607636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6486128-5b78-4a66-afef-d4cc56a8afaa None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:34.607910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6486128-5b78-4a66-afef-d4cc56a8afaa None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:34.608173 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6486128-5b78-4a66-afef-d4cc56a8afaa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:34.615680 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6486128-5b78-4a66-afef-d4cc56a8afaa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:34.617828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6486128-5b78-4a66-afef-d4cc56a8afaa None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:34.631172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-b6486128-5b78-4a66-afef-d4cc56a8afaa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:34.631172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:34.631172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:34.631172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:34.631172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:34.631172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:34.631172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:34.631172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:34.631172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:34.631172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:34.631172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:34.631172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:34.631172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:34.631172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:34.631172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:34.631172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:34.632188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:34.633153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:34.633153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:34.633153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:34.633153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:34.633153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:34.633153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:34.633153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:34.633153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:34.633153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:45:34.633153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:34.633153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 387/791] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Sep 22 14:45:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:34.638528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6486128-5b78-4a66-afef-d4cc56a8afaa None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:34.728041 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b1d11-3406-464c-b959-01f260d9654b None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:34.800058 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932c1fb-25f4-4d07-ab5c-e4a1f4fc1a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:34.800687 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932c1fb-25f4-4d07-ab5c-e4a1f4fc1a98 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:34.800964 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932c1fb-25f4-4d07-ab5c-e4a1f4fc1a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:34.801225 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932c1fb-25f4-4d07-ab5c-e4a1f4fc1a98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:34.816111 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6486128-5b78-4a66-afef-d4cc56a8afaa None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:34.823589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932c1fb-25f4-4d07-ab5c-e4a1f4fc1a98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:34.825032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932c1fb-25f4-4d07-ab5c-e4a1f4fc1a98 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:34.850813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-2932c1fb-25f4-4d07-ab5c-e4a1f4fc1a98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:34.850813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:34.850813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:34.850813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:34.850813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:34.850813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:34.850813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:34.850813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:34.850813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:34.850813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:34.850813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:34.850813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:34.850813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:34.850813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:34.850813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:34.850813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:34.851852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:34.851852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:34.851852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:34.851852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:34.851852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:34.851852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:34.851852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:34.851852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:34.851852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:34.851852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:34.851852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:34.851852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:34.851852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:34.851852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:34.851852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:34.851852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:34.851852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:34.852741 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:34.852741 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:34.852741 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:34.852741 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 405/792] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:34.855456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:34.857585 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595e6b8-b25c-4916-9e30-d943e92cf20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:34.858204 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595e6b8-b25c-4916-9e30-d943e92cf20d None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:34.858483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595e6b8-b25c-4916-9e30-d943e92cf20d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:34.859195 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595e6b8-b25c-4916-9e30-d943e92cf20d None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:34.859562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595e6b8-b25c-4916-9e30-d943e92cf20d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:34.859849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595e6b8-b25c-4916-9e30-d943e92cf20d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:34.861384 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595e6b8-b25c-4916-9e30-d943e92cf20d None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:34.861623 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932c1fb-25f4-4d07-ab5c-e4a1f4fc1a98 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:34.879627 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 388/793] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Sep 22 14:45:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:34.885281 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595e6b8-b25c-4916-9e30-d943e92cf20d None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:34.985692 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:34.988540 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595e6b8-b25c-4916-9e30-d943e92cf20d None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:35.023812 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c50f38-8c55-4b9a-bc3b-65254ba2123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:35.024430 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c50f38-8c55-4b9a-bc3b-65254ba2123e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.024711 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c50f38-8c55-4b9a-bc3b-65254ba2123e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.024982 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c50f38-8c55-4b9a-bc3b-65254ba2123e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.103107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:35.110602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932c1fb-25f4-4d07-ab5c-e4a1f4fc1a98 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:35.128828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c50f38-8c55-4b9a-bc3b-65254ba2123e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:35.130305 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c50f38-8c55-4b9a-bc3b-65254ba2123e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:35.152495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae44a06-2cd3-4817-bf17-1ee42c282aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:35.153118 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae44a06-2cd3-4817-bf17-1ee42c282aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.153394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae44a06-2cd3-4817-bf17-1ee42c282aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.153662 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae44a06-2cd3-4817-bf17-1ee42c282aa6 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.154308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae44a06-2cd3-4817-bf17-1ee42c282aa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:35.154617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae44a06-2cd3-4817-bf17-1ee42c282aa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:35.156014 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae44a06-2cd3-4817-bf17-1ee42c282aa6 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:35.182606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 406/794] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:35.188436 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae44a06-2cd3-4817-bf17-1ee42c282aa6 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:35.301215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 389/795] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:45:35.309545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c50f38-8c55-4b9a-bc3b-65254ba2123e None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:35.348814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:35.357647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae44a06-2cd3-4817-bf17-1ee42c282aa6 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:35.397536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2a0b1-78e0-40a9-b9a2-6f4caeacd0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:35.398164 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2a0b1-78e0-40a9-b9a2-6f4caeacd0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.398441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2a0b1-78e0-40a9-b9a2-6f4caeacd0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.398708 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2a0b1-78e0-40a9-b9a2-6f4caeacd0c1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.406744 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2a0b1-78e0-40a9-b9a2-6f4caeacd0c1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:35.408184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2a0b1-78e0-40a9-b9a2-6f4caeacd0c1 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:35.421288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-cea2a0b1-78e0-40a9-b9a2-6f4caeacd0c1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:35.421288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:35.421288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:35.421288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:35.421288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:35.421288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:35.421288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:35.421288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:35.421288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:35.421288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:35.421288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:35.421288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:35.421288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:35.421288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:35.421288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:35.421288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:35.422873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:35.423828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:35.423828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:35.423828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:35.423828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:35.423828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:35.423828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:35.423828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:35.423828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:35.423828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:35.423828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:35.423828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 407/796] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:35.429766 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2a0b1-78e0-40a9-b9a2-6f4caeacd0c1 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:35.520215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:35.537663 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c50f38-8c55-4b9a-bc3b-65254ba2123e None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:35.599614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2a0b1-78e0-40a9-b9a2-6f4caeacd0c1 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:35.604042 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1026b34-24b7-4787-87f0-6fbf96ad4841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:35.604683 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1026b34-24b7-4787-87f0-6fbf96ad4841 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.604959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1026b34-24b7-4787-87f0-6fbf96ad4841 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.605227 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1026b34-24b7-4787-87f0-6fbf96ad4841 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.620841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1026b34-24b7-4787-87f0-6fbf96ad4841 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:35.622308 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1026b34-24b7-4787-87f0-6fbf96ad4841 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:35.639972 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad7eee-4218-4a8a-9db9-4068173bc938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:35.640633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad7eee-4218-4a8a-9db9-4068173bc938 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.640906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad7eee-4218-4a8a-9db9-4068173bc938 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.641200 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad7eee-4218-4a8a-9db9-4068173bc938 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.641939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad7eee-4218-4a8a-9db9-4068173bc938 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:35.642241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad7eee-4218-4a8a-9db9-4068173bc938 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:35.642931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-f1026b34-24b7-4787-87f0-6fbf96ad4841 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:35.642931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:35.642931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:35.642931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:35.642931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:35.642931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:35.642931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:35.642931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:35.642931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:35.642931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:35.642931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:35.642931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:35.642931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:35.642931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:35.642931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:35.642931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:35.644976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:35.644976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:35.644976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:35.644976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:35.644976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:35.644976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:35.644976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:35.644976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:35.644976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:35.644976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:35.644976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:35.644976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:35.644976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:35.644976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:35.644976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:35.644976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:35.644976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:35.646407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:35.646407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:35.646407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:35.646617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad7eee-4218-4a8a-9db9-4068173bc938 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:35.646701 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 390/797] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:35.646701 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb71e55-9883-412d-8f51-1b357bff3fb5 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.646701 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb71e55-9883-412d-8f51-1b357bff3fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.646701 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb71e55-9883-412d-8f51-1b357bff3fb5 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.653417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 391/798] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:35.654537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb71e55-9883-412d-8f51-1b357bff3fb5 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:35.668737 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 408/799] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Sep 22 14:45:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:35.670255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c6293-870b-40bc-a699-3e384a5d1b6e None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.671068 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c6293-870b-40bc-a699-3e384a5d1b6e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.671442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c6293-870b-40bc-a699-3e384a5d1b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.697255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-dc9c6293-870b-40bc-a699-3e384a5d1b6e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:35.698587 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-dc9c6293-870b-40bc-a699-3e384a5d1b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:35.707504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:35.774346 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb71e55-9883-412d-8f51-1b357bff3fb5 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:35.776748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 409/800] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:35.777674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c6293-870b-40bc-a699-3e384a5d1b6e None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:35.809673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b064e4-cacd-425e-805c-0f47cd2bbf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:35.810596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b064e4-cacd-425e-805c-0f47cd2bbf3f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.810904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b064e4-cacd-425e-805c-0f47cd2bbf3f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.811330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b064e4-cacd-425e-805c-0f47cd2bbf3f None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.812032 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b064e4-cacd-425e-805c-0f47cd2bbf3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:35.812958 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b064e4-cacd-425e-805c-0f47cd2bbf3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:35.814292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b064e4-cacd-425e-805c-0f47cd2bbf3f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:35.826461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 392/801] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:35.827812 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc8f48-ed6f-405a-a3c0-eb9913874887 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.828208 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc8f48-ed6f-405a-a3c0-eb9913874887 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.828555 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc8f48-ed6f-405a-a3c0-eb9913874887 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.829978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 393/802] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:35.831185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97d8c9-1576-4597-b27e-ef243eae965c None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.831537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97d8c9-1576-4597-b27e-ef243eae965c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.831881 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97d8c9-1576-4597-b27e-ef243eae965c None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.833291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 394/803] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:35.835560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97d8c9-1576-4597-b27e-ef243eae965c None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:35.840699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:35.899828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c6293-870b-40bc-a699-3e384a5d1b6e None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:35.936373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31689cd7-6590-4cca-9bcb-2fefd45fafd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:35.937007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31689cd7-6590-4cca-9bcb-2fefd45fafd7 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.937307 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31689cd7-6590-4cca-9bcb-2fefd45fafd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.938091 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31689cd7-6590-4cca-9bcb-2fefd45fafd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.948253 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31689cd7-6590-4cca-9bcb-2fefd45fafd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:35.949716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31689cd7-6590-4cca-9bcb-2fefd45fafd7 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:35.950137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97d8c9-1576-4597-b27e-ef243eae965c None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:35.951905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:35.969107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-31689cd7-6590-4cca-9bcb-2fefd45fafd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:35.969107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:35.969107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:35.969107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:35.969107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:35.969107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:35.969107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:35.969107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:35.969107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:35.969107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:35.969107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:35.969107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:35.969107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:35.969107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:35.969107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:35.969107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:35.970303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:35.970303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:35.970303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:35.970303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:35.970303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:35.970303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:35.970303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:35.970303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:35.970303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:35.970303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:35.970303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:35.970303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:35.970303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:35.970303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:35.970303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:35.970303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:35.970303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:35.971416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:35.971416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:35.971416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:35.971416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 410/804] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:35.972243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47798da1-7607-4a12-9f2e-5c5b85a0cc2e None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.972518 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47798da1-7607-4a12-9f2e-5c5b85a0cc2e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.972788 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47798da1-7607-4a12-9f2e-5c5b85a0cc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:35.992972 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-47798da1-7607-4a12-9f2e-5c5b85a0cc2e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:35.993984 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e6fec-3ad4-43d6-9527-128e92dee5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:35.994183 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-47798da1-7607-4a12-9f2e-5c5b85a0cc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:35.994641 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e6fec-3ad4-43d6-9527-128e92dee5ab None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:35.994934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e6fec-3ad4-43d6-9527-128e92dee5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:35.995258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e6fec-3ad4-43d6-9527-128e92dee5ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.002330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e6fec-3ad4-43d6-9527-128e92dee5ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.003790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e6fec-3ad4-43d6-9527-128e92dee5ab None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.015182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-ed0e6fec-3ad4-43d6-9527-128e92dee5ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:36.015182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:36.015182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:36.015182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:36.015182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:36.015182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:36.015182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:36.015182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:36.015182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:36.015182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:36.015182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:36.015182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:36.015182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:36.015182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:36.015182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:36.016142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:36.016142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:36.016142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:36.016142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:36.016142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:36.016142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:36.016142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:36.016142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:36.016142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:36.016142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:36.016142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:36.016142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:36.016142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:36.016142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:36.016142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:36.016142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:36.016142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:36.017067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:36.017067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:36.017067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:36.017067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:36.017067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:36.017067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:36.017067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:36.017067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:36.017067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 395/805] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:36.017543 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bf16f-be88-4f37-84a6-f3213625e93d None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.018079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bf16f-be88-4f37-84a6-f3213625e93d None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.018079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094bf16f-be88-4f37-84a6-f3213625e93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.037036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-094bf16f-be88-4f37-84a6-f3213625e93d None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:36.038239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-094bf16f-be88-4f37-84a6-f3213625e93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:36.069462 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 411/806] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:36.070647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47798da1-7607-4a12-9f2e-5c5b85a0cc2e None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.114934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:36.127581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 396/807] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:36.128673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094bf16f-be88-4f37-84a6-f3213625e93d None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.178166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47798da1-7607-4a12-9f2e-5c5b85a0cc2e None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.207080 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:36.218990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255d2ed-cf24-48a1-99e9-822147461188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.219612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255d2ed-cf24-48a1-99e9-822147461188 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.219885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255d2ed-cf24-48a1-99e9-822147461188 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.220152 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255d2ed-cf24-48a1-99e9-822147461188 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.220856 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255d2ed-cf24-48a1-99e9-822147461188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:36.221158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255d2ed-cf24-48a1-99e9-822147461188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.222820 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255d2ed-cf24-48a1-99e9-822147461188 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.240454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 412/808] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:36.240454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255d2ed-cf24-48a1-99e9-822147461188 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.354362 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255d2ed-cf24-48a1-99e9-822147461188 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.393777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094bf16f-be88-4f37-84a6-f3213625e93d None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.399308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb32858-c9ce-4ddc-9d08-cc395fb4b5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.399956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb32858-c9ce-4ddc-9d08-cc395fb4b5ce None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.400232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb32858-c9ce-4ddc-9d08-cc395fb4b5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.400502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb32858-c9ce-4ddc-9d08-cc395fb4b5ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.412487 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb32858-c9ce-4ddc-9d08-cc395fb4b5ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:36.417782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-eeb32858-c9ce-4ddc-9d08-cc395fb4b5ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:36.417782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:36.417782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:36.417782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:36.417782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:36.417782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:36.417782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:36.417782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:36.417782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:36.417782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:36.417782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:36.417782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:36.417782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:36.417782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:36.417782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:36.417782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:36.418774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:36.418774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:36.418774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:36.418774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:36.418774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:36.418774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:36.418774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:36.418774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:36.418774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:36.418774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:36.418774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:36.418774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:36.418774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:36.418774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:36.418774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:36.418774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:36.419609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:36.419609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:36.419609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:36.419609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:36.419609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:36.419609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:36.419609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:36.419609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:36.419609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:36.419609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:36.419609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:36.419609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:36.419609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 413/809] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:36.420299 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb32858-c9ce-4ddc-9d08-cc395fb4b5ce None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.434169 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edadadb-18c4-47fe-924c-283216ebe566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.434675 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edadadb-18c4-47fe-924c-283216ebe566 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.434973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edadadb-18c4-47fe-924c-283216ebe566 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.435319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edadadb-18c4-47fe-924c-283216ebe566 None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.435956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edadadb-18c4-47fe-924c-283216ebe566 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:36.436238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edadadb-18c4-47fe-924c-283216ebe566 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.437670 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edadadb-18c4-47fe-924c-283216ebe566 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.449222 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 397/810] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:36.450026 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edadadb-18c4-47fe-924c-283216ebe566 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.466381 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:36.503911 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:36.522010 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb32858-c9ce-4ddc-9d08-cc395fb4b5ce None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.551301 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edadadb-18c4-47fe-924c-283216ebe566 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.555234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534bbc73-59eb-4ce5-b6e0-24ce1c84d50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.555861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bbc73-59eb-4ce5-b6e0-24ce1c84d50a None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.556146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bbc73-59eb-4ce5-b6e0-24ce1c84d50a None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.556417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534bbc73-59eb-4ce5-b6e0-24ce1c84d50a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.563148 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534bbc73-59eb-4ce5-b6e0-24ce1c84d50a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.564586 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534bbc73-59eb-4ce5-b6e0-24ce1c84d50a None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-534bbc73-59eb-4ce5-b6e0-24ce1c84d50a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:36.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:36.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:36.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:36.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:36.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:36.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:36.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:36.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:36.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:36.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:36.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:36.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:36.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:36.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:36.576814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:36.576814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:36.576814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:36.576814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:36.576814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:36.576814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:36.576814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:36.576814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:36.576814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:36.576814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:36.576814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:36.576814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:36.576814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:36.576814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:36.576814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:36.576814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:36.576814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:36.577732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:36.577732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:36.577732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:36.577732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:36.577732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:36.577732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:36.577732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:36.577732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:36.577732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 414/811] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:36.578777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534bbc73-59eb-4ce5-b6e0-24ce1c84d50a None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.588992 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a651495-1bd0-48cd-b7ea-5bff0508ae1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.589462 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a651495-1bd0-48cd-b7ea-5bff0508ae1d None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.589739 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a651495-1bd0-48cd-b7ea-5bff0508ae1d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.590008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a651495-1bd0-48cd-b7ea-5bff0508ae1d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.590590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a651495-1bd0-48cd-b7ea-5bff0508ae1d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.591971 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a651495-1bd0-48cd-b7ea-5bff0508ae1d None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.601579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-8a651495-1bd0-48cd-b7ea-5bff0508ae1d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:36.601579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:36.601579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:36.601579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:36.601579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:36.601579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:36.601579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:36.601579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:36.601579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:36.601579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:36.601579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:36.601579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:36.601579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:36.601579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:36.601579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:36.601579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:36.601579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:36.602623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:36.602623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:36.602623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:36.602623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:36.602623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:36.602623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:36.602623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:36.602623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:36.602623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:36.602623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:36.602623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:36.602623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:36.602623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:36.602623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:36.602623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:36.602623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:36.603873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:36.603873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:36.603873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:36.603873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:36.603873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:36.603873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:36.603873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:36.603873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:36.603873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:36.603873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:36.603873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:36.603873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:36.603873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 14:45:36.603873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:36.603873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 398/812] 10.222.0.26 () {56 vars in 1129 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:36.604819 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a651495-1bd0-48cd-b7ea-5bff0508ae1d None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.683470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534bbc73-59eb-4ce5-b6e0-24ce1c84d50a None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.685876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:36.703183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a651495-1bd0-48cd-b7ea-5bff0508ae1d None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.719625 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fa8be-64a8-4f8c-b523-d9cf26d7d199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.720248 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fa8be-64a8-4f8c-b523-d9cf26d7d199 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.720521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fa8be-64a8-4f8c-b523-d9cf26d7d199 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.720617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:36.720788 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fa8be-64a8-4f8c-b523-d9cf26d7d199 None admin] PATH_INFO: `/v3/services/network` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.721490 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340fa8be-64a8-4f8c-b523-d9cf26d7d199 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.723092 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340fa8be-64a8-4f8c-b523-d9cf26d7d199 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.732768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-340fa8be-64a8-4f8c-b523-d9cf26d7d199 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:36.732768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:36.732768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:36.732768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:36.732768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:36.732768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:36.732768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:36.732768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:36.732768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:36.732768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:36.732768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:36.732768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:36.732768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:36.732768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:36.732768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:36.732768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:36.732768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:36.733994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:36.733994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:36.733994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:36.733994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:36.733994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:36.733994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:36.733994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:36.733994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:36.733994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:36.733994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:36.733994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:36.733994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:36.733994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:36.733994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:36.733994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:36.733994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:36.735630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:36.735630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:36.735630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:36.735630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:36.735630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:36.735630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:36.735630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:36.735630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:36.735630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:36.735630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:36.735630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:36.735630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:36.735630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 14:45:36.735630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:36.735630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 415/813] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/services/network => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:36.736608 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fa8be-64a8-4f8c-b523-d9cf26d7d199 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.744497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd4dc3-9121-4db3-9a86-296edd78e457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.745105 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd4dc3-9121-4db3-9a86-296edd78e457 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.745380 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd4dc3-9121-4db3-9a86-296edd78e457 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.745647 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd4dc3-9121-4db3-9a86-296edd78e457 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.746309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd4dc3-9121-4db3-9a86-296edd78e457 None admin] RBAC: Adding query filter params (name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:36.746635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd4dc3-9121-4db3-9a86-296edd78e457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.748025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd4dc3-9121-4db3-9a86-296edd78e457 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.756523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 399/814] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:36.757316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd4dc3-9121-4db3-9a86-296edd78e457 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.838661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fa8be-64a8-4f8c-b523-d9cf26d7d199 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.859149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd4dc3-9121-4db3-9a86-296edd78e457 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:36.871852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71261ab-c6dc-4281-8333-aaacb4e4886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.872450 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71261ab-c6dc-4281-8333-aaacb4e4886e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.872725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71261ab-c6dc-4281-8333-aaacb4e4886e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.872995 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71261ab-c6dc-4281-8333-aaacb4e4886e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.873710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71261ab-c6dc-4281-8333-aaacb4e4886e None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=0c9be41c39494e54a14398a1f03c4d69) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:36.874023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71261ab-c6dc-4281-8333-aaacb4e4886e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.876017 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71261ab-c6dc-4281-8333-aaacb4e4886e None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.894807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 416/815] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/role_assignments?user.id=0c9be41c39494e54a14398a1f03c4d69&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=32c8a81e339a462e973e76bb528ef35a => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:36.894807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71261ab-c6dc-4281-8333-aaacb4e4886e None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.898033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f043f4-deb7-49cd-b890-0cca7324e36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:36.898661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f043f4-deb7-49cd-b890-0cca7324e36f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:36.898986 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f043f4-deb7-49cd-b890-0cca7324e36f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:36.899257 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f043f4-deb7-49cd-b890-0cca7324e36f None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:36.899959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f043f4-deb7-49cd-b890-0cca7324e36f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:36.900265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f043f4-deb7-49cd-b890-0cca7324e36f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:36.901753 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f043f4-deb7-49cd-b890-0cca7324e36f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:36.919245 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 400/816] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:36.920373 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f043f4-deb7-49cd-b890-0cca7324e36f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:36.942887 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:36.968197 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:37.026426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71261ab-c6dc-4281-8333-aaacb4e4886e None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.039269 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f043f4-deb7-49cd-b890-0cca7324e36f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.068709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a7779-734d-44c4-b7e1-3c21fccc9183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.069328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a7779-734d-44c4-b7e1-3c21fccc9183 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.069608 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a7779-734d-44c4-b7e1-3c21fccc9183 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.070970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a7779-734d-44c4-b7e1-3c21fccc9183 None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.070970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a7779-734d-44c4-b7e1-3c21fccc9183 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:37.070970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a7779-734d-44c4-b7e1-3c21fccc9183 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.073101 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a7779-734d-44c4-b7e1-3c21fccc9183 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:37.079822 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 417/817] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:37.079822 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a7779-734d-44c4-b7e1-3c21fccc9183 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.080060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b0539-8e69-4748-bd9d-5d5da1c7fc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.080424 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b0539-8e69-4748-bd9d-5d5da1c7fc2f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.080755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b0539-8e69-4748-bd9d-5d5da1c7fc2f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.080970 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b0539-8e69-4748-bd9d-5d5da1c7fc2f None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.081592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b0539-8e69-4748-bd9d-5d5da1c7fc2f None admin] RBAC: Adding query filter params (name=network) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:37.081890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b0539-8e69-4748-bd9d-5d5da1c7fc2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.083284 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b0539-8e69-4748-bd9d-5d5da1c7fc2f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:37.087950 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 401/818] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:37.088728 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b0539-8e69-4748-bd9d-5d5da1c7fc2f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.185530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:37.192668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a7779-734d-44c4-b7e1-3c21fccc9183 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.205393 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b0539-8e69-4748-bd9d-5d5da1c7fc2f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.207060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:37.231618 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341e91d-b9d7-4395-a045-21bd159788e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.232176 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341e91d-b9d7-4395-a045-21bd159788e9 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.232453 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341e91d-b9d7-4395-a045-21bd159788e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.232718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341e91d-b9d7-4395-a045-21bd159788e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.241838 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2aa4d3-8eca-4d30-be25-57fd19ee2da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.242499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2aa4d3-8eca-4d30-be25-57fd19ee2da9 None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.242799 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2aa4d3-8eca-4d30-be25-57fd19ee2da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.243068 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2aa4d3-8eca-4d30-be25-57fd19ee2da9 None admin] PATH_INFO: `/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/0a265623ae734936a340c97df852e227/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.243972 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341e91d-b9d7-4395-a045-21bd159788e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.244970 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2aa4d3-8eca-4d30-be25-57fd19ee2da9 None admin] RBAC: Authorizing `identity:create_grant(project_id=43bc5871e4bf441b874474bcae4e5385, user_id=0a265623ae734936a340c97df852e227, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.245401 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341e91d-b9d7-4395-a045-21bd159788e9 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:37.246379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2aa4d3-8eca-4d30-be25-57fd19ee2da9 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:37.270766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 402/819] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:37 2022] PUT /identity/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/0a265623ae734936a340c97df852e227/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:45:37.270980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-b341e91d-b9d7-4395-a045-21bd159788e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:37.270980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:37.270980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:37.270980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:37.270980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:37.270980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:37.270980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:37.270980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:37.270980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:37.270980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:37.270980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:37.270980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:37.270980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:37.270980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:37.270980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:37.270980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:37.272057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:37.272057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:37.272057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:37.272057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:37.272057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:37.272057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:37.272057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:37.272057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:37.272057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:37.272057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:37.272057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:37.272057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:37.272057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:37.272057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:37.272057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:37.272057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:37.272057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:37.273121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:37.273121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:37.273121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:37.273335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2aa4d3-8eca-4d30-be25-57fd19ee2da9 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.273419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 418/820] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:37.273419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341e91d-b9d7-4395-a045-21bd159788e9 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.410467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341e91d-b9d7-4395-a045-21bd159788e9 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.445144 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2aa4d3-8eca-4d30-be25-57fd19ee2da9 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.454694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe783d-9672-4b8f-abb7-09034c73c750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.455559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe783d-9672-4b8f-abb7-09034c73c750 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.455924 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe783d-9672-4b8f-abb7-09034c73c750 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.456294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe783d-9672-4b8f-abb7-09034c73c750 None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.457246 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe783d-9672-4b8f-abb7-09034c73c750 None admin] RBAC: Adding query filter params (type=network) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:37.457402 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe783d-9672-4b8f-abb7-09034c73c750 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.458912 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe783d-9672-4b8f-abb7-09034c73c750 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:37.465808 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 419/821] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:37.466873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe783d-9672-4b8f-abb7-09034c73c750 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.485790 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:37.498889 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cca0c-c9f7-43e4-9861-5428f257cad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.500259 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cca0c-c9f7-43e4-9861-5428f257cad0 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.500684 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cca0c-c9f7-43e4-9861-5428f257cad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.501089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cca0c-c9f7-43e4-9861-5428f257cad0 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.501898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cca0c-c9f7-43e4-9861-5428f257cad0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:37.502197 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cca0c-c9f7-43e4-9861-5428f257cad0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.505180 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cca0c-c9f7-43e4-9861-5428f257cad0 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:37.514787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 403/822] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Sep 22 14:45:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:37.514787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cca0c-c9f7-43e4-9861-5428f257cad0 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.544800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:37.600502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe783d-9672-4b8f-abb7-09034c73c750 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.645216 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cca0c-c9f7-43e4-9861-5428f257cad0 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.647087 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f9dc7-bc6a-4eb0-940b-d82e9e62eef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.648869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f9dc7-bc6a-4eb0-940b-d82e9e62eef1 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.649337 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f9dc7-bc6a-4eb0-940b-d82e9e62eef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.649810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f9dc7-bc6a-4eb0-940b-d82e9e62eef1 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.650646 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f9dc7-bc6a-4eb0-940b-d82e9e62eef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:37.651152 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f9dc7-bc6a-4eb0-940b-d82e9e62eef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.653274 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f9dc7-bc6a-4eb0-940b-d82e9e62eef1 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:37.673918 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 420/823] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:37.673918 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f9dc7-bc6a-4eb0-940b-d82e9e62eef1 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.698957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a3d41-bf14-4ecf-a0cd-2e74d1432e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.699571 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a3d41-bf14-4ecf-a0cd-2e74d1432e26 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.699843 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a3d41-bf14-4ecf-a0cd-2e74d1432e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.700108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a3d41-bf14-4ecf-a0cd-2e74d1432e26 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.700771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a3d41-bf14-4ecf-a0cd-2e74d1432e26 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.702142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a3d41-bf14-4ecf-a0cd-2e74d1432e26 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:37.738837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 404/824] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:45:37 2022] POST /identity/v3/endpoints => generated 337 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:37.738837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a3d41-bf14-4ecf-a0cd-2e74d1432e26 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:37.832126 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:37.901377 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f9dc7-bc6a-4eb0-940b-d82e9e62eef1 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.927508 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:37.964893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a3d41-bf14-4ecf-a0cd-2e74d1432e26 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:37.967826 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421bb6d-a5c5-4f28-b4be-96fb7cea0338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:37.969369 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421bb6d-a5c5-4f28-b4be-96fb7cea0338 None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:37.969663 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421bb6d-a5c5-4f28-b4be-96fb7cea0338 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:37.969937 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421bb6d-a5c5-4f28-b4be-96fb7cea0338 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:37.970608 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421bb6d-a5c5-4f28-b4be-96fb7cea0338 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:37.972371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421bb6d-a5c5-4f28-b4be-96fb7cea0338 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:38.003948 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 421/825] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:45:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:38.019017 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de5cf9-efec-43fb-a7ac-42aa6d82f4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:38.019947 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de5cf9-efec-43fb-a7ac-42aa6d82f4ab None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:38.020230 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de5cf9-efec-43fb-a7ac-42aa6d82f4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:38.020500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de5cf9-efec-43fb-a7ac-42aa6d82f4ab None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:38.030298 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de5cf9-efec-43fb-a7ac-42aa6d82f4ab None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:38.032379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de5cf9-efec-43fb-a7ac-42aa6d82f4ab None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:38.045396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-69de5cf9-efec-43fb-a7ac-42aa6d82f4ab None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:38.045396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:38.045396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:38.045396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:38.045396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:38.045396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:38.045396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:38.045396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:38.045396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:38.045396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:38.045396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:38.045396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:38.045396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:38.045396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:38.045396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:38.045396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:38.046406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:38.047445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:38.047445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:38.047445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:38.047445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:38.047445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:38.047445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:38.047445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:38.047445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:38.047445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:38.047445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:38.050923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 405/826] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Sep 22 14:45:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:38.053232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421bb6d-a5c5-4f28-b4be-96fb7cea0338 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:38.189227 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421bb6d-a5c5-4f28-b4be-96fb7cea0338 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:38.223472 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:38.237024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41e429-77f2-4481-bc01-53e6b0bb7641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:38.237641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41e429-77f2-4481-bc01-53e6b0bb7641 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:38.237923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41e429-77f2-4481-bc01-53e6b0bb7641 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:38.238195 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41e429-77f2-4481-bc01-53e6b0bb7641 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:38.239084 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41e429-77f2-4481-bc01-53e6b0bb7641 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:38.239395 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41e429-77f2-4481-bc01-53e6b0bb7641 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:38.240867 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41e429-77f2-4481-bc01-53e6b0bb7641 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:38.263378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 422/827] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Sep 22 14:45:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:38.269258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de5cf9-efec-43fb-a7ac-42aa6d82f4ab None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:38.378150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de5cf9-efec-43fb-a7ac-42aa6d82f4ab None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:38.414025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fb2c5-83d1-4133-83de-4ee683e41f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:38.414655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fb2c5-83d1-4133-83de-4ee683e41f40 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:38.414956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fb2c5-83d1-4133-83de-4ee683e41f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:38.415224 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010fb2c5-83d1-4133-83de-4ee683e41f40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:38.417029 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:38.429345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fb2c5-83d1-4133-83de-4ee683e41f40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:38.430824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010fb2c5-83d1-4133-83de-4ee683e41f40 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:38.446067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-010fb2c5-83d1-4133-83de-4ee683e41f40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:38.446067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:38.446067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:38.446067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:38.446067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:38.446067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:38.446067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:38.446067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:38.446067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:38.446067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:38.446067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:38.446067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:38.446067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:38.446067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:38.446067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:38.446067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:38.447097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:38.447097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:38.447097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:38.447097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:38.447097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:38.447097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:38.447097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:38.447097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:38.447097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:38.447097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:38.447097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:38.447097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:38.447097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:38.447097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:38.447097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:38.447097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:38.447097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:38.448270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:38.448270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:38.448270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:38.448270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 406/828] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:38.455483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41e429-77f2-4481-bc01-53e6b0bb7641 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:38.566085 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41e429-77f2-4481-bc01-53e6b0bb7641 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:38.602791 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86464efb-3572-4de1-a795-97ea0881af5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:38.603401 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86464efb-3572-4de1-a795-97ea0881af5a None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:38.603675 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86464efb-3572-4de1-a795-97ea0881af5a None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:38.603944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86464efb-3572-4de1-a795-97ea0881af5a None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:38.604581 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86464efb-3572-4de1-a795-97ea0881af5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:38.604880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86464efb-3572-4de1-a795-97ea0881af5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:38.606230 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86464efb-3572-4de1-a795-97ea0881af5a None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:38.618705 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 423/829] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:38.624941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fb2c5-83d1-4133-83de-4ee683e41f40 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:38.731851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010fb2c5-83d1-4133-83de-4ee683e41f40 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:38.770049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760694f4-434b-4819-948c-a46e98094669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:38.771129 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760694f4-434b-4819-948c-a46e98094669 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:38.771640 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760694f4-434b-4819-948c-a46e98094669 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:38.772135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760694f4-434b-4819-948c-a46e98094669 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:38.781737 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760694f4-434b-4819-948c-a46e98094669 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:38.783980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760694f4-434b-4819-948c-a46e98094669 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:38.796079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-760694f4-434b-4819-948c-a46e98094669 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:38.796079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:38.796079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:38.796079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:38.796079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:38.796079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:38.796079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:38.796079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:38.796079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:38.796079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:38.796079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:38.796079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:38.796079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:38.796079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:38.796079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:38.797132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:38.797132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:38.797132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:38.797132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:38.797132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:38.797132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:38.797132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:38.797132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:38.797132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:38.797132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:38.797132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:38.797132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:38.797132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:38.797132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:38.797132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:38.797132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:38.797132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:38.798121 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:38.798121 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:38.798121 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:38.798121 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:38.798121 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:38.798121 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:38.798121 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:38.798121 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:38.804973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 407/830] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:38.806606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86464efb-3572-4de1-a795-97ea0881af5a None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:38.859800 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:38.923644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86464efb-3572-4de1-a795-97ea0881af5a None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:38.960962 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef2c9e1-aaaf-46ee-87b7-2fd4fd21314d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:38.961986 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2c9e1-aaaf-46ee-87b7-2fd4fd21314d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:38.962486 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2c9e1-aaaf-46ee-87b7-2fd4fd21314d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:38.962998 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef2c9e1-aaaf-46ee-87b7-2fd4fd21314d None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:38.963893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2c9e1-aaaf-46ee-87b7-2fd4fd21314d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:38.964455 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2c9e1-aaaf-46ee-87b7-2fd4fd21314d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:38.966152 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef2c9e1-aaaf-46ee-87b7-2fd4fd21314d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:38.985541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 424/831] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:38.987426 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760694f4-434b-4819-948c-a46e98094669 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:39.036786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:39.096100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760694f4-434b-4819-948c-a46e98094669 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:39.132481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19691c5a-ec35-49f8-bfe2-5771f389a2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:39.133498 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19691c5a-ec35-49f8-bfe2-5771f389a2df None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:39.133993 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19691c5a-ec35-49f8-bfe2-5771f389a2df None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:39.134470 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19691c5a-ec35-49f8-bfe2-5771f389a2df None admin] PATH_INFO: `/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/6256407d9df648d5bbe33495cb9a8b36/roles/35bdd5122aae45ebbe44158a433d2f99` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:39.150578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19691c5a-ec35-49f8-bfe2-5771f389a2df None admin] RBAC: Authorizing `identity:create_grant(project_id=43bc5871e4bf441b874474bcae4e5385, user_id=6256407d9df648d5bbe33495cb9a8b36, role_id=35bdd5122aae45ebbe44158a433d2f99)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:39.152456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19691c5a-ec35-49f8-bfe2-5771f389a2df None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:39.174885 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 408/832] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:38 2022] PUT /identity/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/6256407d9df648d5bbe33495cb9a8b36/roles/35bdd5122aae45ebbe44158a433d2f99 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:45:39.349842 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e287287-f454-495f-a5c1-9b8853a75f4c None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:39.349842 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e287287-f454-495f-a5c1-9b8853a75f4c None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:39.349842 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e287287-f454-495f-a5c1-9b8853a75f4c None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:39.351078 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 425/833] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:39.360041 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e49d3-47ce-4b3e-b670-47b2f1d9a754 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:39.360041 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e49d3-47ce-4b3e-b670-47b2f1d9a754 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:39.360041 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46e49d3-47ce-4b3e-b670-47b2f1d9a754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:39.377285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-b46e49d3-47ce-4b3e-b670-47b2f1d9a754 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:39.378837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b46e49d3-47ce-4b3e-b670-47b2f1d9a754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:39.456990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 409/834] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:39.570001 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d122cd-9dc6-4e2f-b158-8cff710391d5 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:39.570326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d122cd-9dc6-4e2f-b158-8cff710391d5 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:39.570546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d122cd-9dc6-4e2f-b158-8cff710391d5 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:39.572129 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 426/835] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:39.577592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48853bc7-3880-4a6c-b6a1-8858405d1f76 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:39.578439 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48853bc7-3880-4a6c-b6a1-8858405d1f76 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:39.578709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48853bc7-3880-4a6c-b6a1-8858405d1f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:39.597875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-48853bc7-3880-4a6c-b6a1-8858405d1f76 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:39.599431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-48853bc7-3880-4a6c-b6a1-8858405d1f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:39.620168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:39.677743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 410/836] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:39.750559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d122cd-9dc6-4e2f-b158-8cff710391d5 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:39.863476 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d122cd-9dc6-4e2f-b158-8cff710391d5 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:39.866427 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:39.915926 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e35b0-e06d-4a64-a864-1fdde2da7cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:39.916939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e35b0-e06d-4a64-a864-1fdde2da7cfa None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:39.917420 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e35b0-e06d-4a64-a864-1fdde2da7cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:39.917893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e35b0-e06d-4a64-a864-1fdde2da7cfa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:39.918672 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e35b0-e06d-4a64-a864-1fdde2da7cfa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:39.921430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e35b0-e06d-4a64-a864-1fdde2da7cfa None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:39.940700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-e13e35b0-e06d-4a64-a864-1fdde2da7cfa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:39.940700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:39.940700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:39.940700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:39.940700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:39.940700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:39.940700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:39.940700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:39.940700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:39.940700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:39.940700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:39.940700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:39.940700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:39.940700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:39.940700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:39.940700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:39.941781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:39.941781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:39.941781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:39.941781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:39.941781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:39.941781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:39.941781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:39.941781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:39.941781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:39.941781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:39.941781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:39.941781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:39.941781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:39.941781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:39.941781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:39.941781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:39.941781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:39.942813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:39.942813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:39.942813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:39.942813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:39.942813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:39.942813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:39.942813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:39.942813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:39.942813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:39.942813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:39.942813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:39.942813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:39.942813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:39.942813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:39.945915 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 427/837] 10.222.0.26 () {56 vars in 1132 bytes} [Thu Sep 22 14:45:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:39.952672 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48853bc7-3880-4a6c-b6a1-8858405d1f76 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:39.959206 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e35b0-e06d-4a64-a864-1fdde2da7cfa None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:40.101581 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e35b0-e06d-4a64-a864-1fdde2da7cfa None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:40.141359 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a55b73-f7ac-4f94-9c16-523de0ac8f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:40.142357 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a55b73-f7ac-4f94-9c16-523de0ac8f6f None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:40.143379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a55b73-f7ac-4f94-9c16-523de0ac8f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:40.143879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a55b73-f7ac-4f94-9c16-523de0ac8f6f None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:40.144792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a55b73-f7ac-4f94-9c16-523de0ac8f6f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:40.145359 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a55b73-f7ac-4f94-9c16-523de0ac8f6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:40.146948 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a55b73-f7ac-4f94-9c16-523de0ac8f6f None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:40.158240 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 428/838] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:40.159974 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a55b73-f7ac-4f94-9c16-523de0ac8f6f None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:40.173616 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48853bc7-3880-4a6c-b6a1-8858405d1f76 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:40.201428 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:40.223440 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424177c-fa33-444b-ab85-9cd4312ad8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:40.224078 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424177c-fa33-444b-ab85-9cd4312ad8df None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:40.224351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424177c-fa33-444b-ab85-9cd4312ad8df None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:40.224615 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424177c-fa33-444b-ab85-9cd4312ad8df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:40.240377 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424177c-fa33-444b-ab85-9cd4312ad8df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:40.245355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-5424177c-fa33-444b-ab85-9cd4312ad8df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:40.245355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:40.245355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:40.245355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:40.245355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:40.245355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:40.245355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:40.245355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:40.245355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:40.245355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:40.245355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:40.245355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:40.245355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:40.245355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:40.245355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:40.245355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:40.246280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:40.246280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:40.246280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:40.246280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:40.246280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:40.246280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:40.246280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:40.246280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:40.246280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:40.246280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:40.246280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:40.246280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:40.246280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:40.246280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:40.246280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:40.246280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:40.247199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:40.247199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:40.247199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:40.247199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:40.247199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:40.247199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:40.247199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:40.247199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:40.247199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:40.247199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:40.247199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:45:40.247199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:40.252192 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 411/839] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:40.256036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424177c-fa33-444b-ab85-9cd4312ad8df None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:40.276653 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:40.316342 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a55b73-f7ac-4f94-9c16-523de0ac8f6f None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:40.381358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994caf84-4238-4c42-9032-9e969b59cabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:40.382483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994caf84-4238-4c42-9032-9e969b59cabf None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:40.383041 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994caf84-4238-4c42-9032-9e969b59cabf None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:40.383563 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994caf84-4238-4c42-9032-9e969b59cabf None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:40.384628 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994caf84-4238-4c42-9032-9e969b59cabf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:40.385228 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994caf84-4238-4c42-9032-9e969b59cabf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:40.387836 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994caf84-4238-4c42-9032-9e969b59cabf None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:40.411308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 429/840] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:40 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:40.426464 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424177c-fa33-444b-ab85-9cd4312ad8df None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:40.465439 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deecf22-924d-4d86-9fc3-4cc9db361fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:40.466461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deecf22-924d-4d86-9fc3-4cc9db361fbb None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:40.466976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deecf22-924d-4d86-9fc3-4cc9db361fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:40.467451 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deecf22-924d-4d86-9fc3-4cc9db361fbb None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:40.468321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deecf22-924d-4d86-9fc3-4cc9db361fbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:40.468879 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deecf22-924d-4d86-9fc3-4cc9db361fbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:40.470124 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f14f7f-3a8d-4cc1-ba8b-62c9cbe0351b None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:40.470414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f14f7f-3a8d-4cc1-ba8b-62c9cbe0351b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:40.470697 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f14f7f-3a8d-4cc1-ba8b-62c9cbe0351b None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:40.471110 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deecf22-924d-4d86-9fc3-4cc9db361fbb None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:40.476510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 430/841] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:40.478493 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4b51c-9f62-4949-a61d-22050a094312 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:40.478644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4b51c-9f62-4949-a61d-22050a094312 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:40.478940 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4b51c-9f62-4949-a61d-22050a094312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:40.482902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 412/842] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:40.492867 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deecf22-924d-4d86-9fc3-4cc9db361fbb None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:40.512083 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-62d4b51c-9f62-4949-a61d-22050a094312 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:40.513793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-62d4b51c-9f62-4949-a61d-22050a094312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:40.607529 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 431/843] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:40.660011 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:40.683791 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deecf22-924d-4d86-9fc3-4cc9db361fbb None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:40.719398 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fcc17-05ae-4599-874b-18af27189f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:40.720035 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fcc17-05ae-4599-874b-18af27189f2a None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:40.720313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fcc17-05ae-4599-874b-18af27189f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:40.720579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fcc17-05ae-4599-874b-18af27189f2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:40.732031 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52fcc17-05ae-4599-874b-18af27189f2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:40.733537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52fcc17-05ae-4599-874b-18af27189f2a None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:40.748133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-f52fcc17-05ae-4599-874b-18af27189f2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:40.748133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:40.748133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:40.748133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:40.748133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:40.748133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:40.748133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:40.748133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:40.748133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:40.748133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:40.748133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:40.748133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:40.748133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:40.748133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:40.748133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:40.748133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:40.749124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:40.749124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:40.749124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:40.749124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:40.749124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:40.749124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:40.749124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:40.749124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:40.749124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:40.749124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:40.749124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:40.749124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:40.749124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:40.749124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:40.749124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:40.749124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:40.749124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:40.750575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:40.750575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:40.750575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:40.750575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 413/844] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:40.755837 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4b51c-9f62-4949-a61d-22050a094312 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:40.785044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:40.871140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4b51c-9f62-4949-a61d-22050a094312 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:40.880974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fcc17-05ae-4599-874b-18af27189f2a None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:40.915601 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b373f6-fda5-4c2f-9664-8511f0ccaea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:40.916461 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b373f6-fda5-4c2f-9664-8511f0ccaea8 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:40.917086 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b373f6-fda5-4c2f-9664-8511f0ccaea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:40.917508 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b373f6-fda5-4c2f-9664-8511f0ccaea8 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:40.919083 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b373f6-fda5-4c2f-9664-8511f0ccaea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:40.919546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b373f6-fda5-4c2f-9664-8511f0ccaea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:40.921062 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b373f6-fda5-4c2f-9664-8511f0ccaea8 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:40.936733 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 432/845] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:40.944359 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b373f6-fda5-4c2f-9664-8511f0ccaea8 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:40.993048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fcc17-05ae-4599-874b-18af27189f2a None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:41.031772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:41.039765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261089e1-97aa-4274-bc7f-81eb1268d0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:41.040474 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261089e1-97aa-4274-bc7f-81eb1268d0cb None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.040755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261089e1-97aa-4274-bc7f-81eb1268d0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.041053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261089e1-97aa-4274-bc7f-81eb1268d0cb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.042097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261089e1-97aa-4274-bc7f-81eb1268d0cb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:41.043693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261089e1-97aa-4274-bc7f-81eb1268d0cb None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:41.052262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:41.057924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-261089e1-97aa-4274-bc7f-81eb1268d0cb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:41.057924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:41.057924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:41.057924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:41.057924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:41.057924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:41.057924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:41.057924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:41.057924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:41.057924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:41.057924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:41.057924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:41.057924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:41.057924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:41.057924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:41.057924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:41.057924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:41.058974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:41.058974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:41.058974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:41.058974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:41.058974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:41.058974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:41.058974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:41.058974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:41.058974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:41.058974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:41.058974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:41.058974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:41.058974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:41.058974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:41.058974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:41.058974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:41.059803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:41.059803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:41.059803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:41.059803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:41.059803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:41.059803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:41.059803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:41.059803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:41.059803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:41.059803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:41.059803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:41.059803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:41.059803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:41.059803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:41.062814 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 414/846] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:41.066814 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261089e1-97aa-4274-bc7f-81eb1268d0cb None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:41.078896 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b373f6-fda5-4c2f-9664-8511f0ccaea8 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:41.121198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a38bae-347b-4969-8f0b-73958996dd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:41.121838 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a38bae-347b-4969-8f0b-73958996dd53 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.122117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a38bae-347b-4969-8f0b-73958996dd53 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.122389 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a38bae-347b-4969-8f0b-73958996dd53 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.132869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a38bae-347b-4969-8f0b-73958996dd53 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:41.134316 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a38bae-347b-4969-8f0b-73958996dd53 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:41.148210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-17a38bae-347b-4969-8f0b-73958996dd53 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:41.148210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:41.148210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:41.148210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:41.148210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:41.148210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:41.148210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:41.148210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:41.148210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:41.148210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:41.148210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:41.148210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:41.148210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:41.148210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:41.148210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:41.148210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:41.149207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:41.150171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:41.150171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:41.150171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:41.150171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:41.150171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:41.150171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:41.150171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:41.150171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:41.150171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 14:45:41.150171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:41.154329 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 433/847] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:45:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:41.155664 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a38bae-347b-4969-8f0b-73958996dd53 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:41.210171 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261089e1-97aa-4274-bc7f-81eb1268d0cb None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:41.257823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85769d49-9122-48b5-83bf-90f9fe6fd81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:41.258880 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85769d49-9122-48b5-83bf-90f9fe6fd81d None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.259374 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85769d49-9122-48b5-83bf-90f9fe6fd81d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.260387 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85769d49-9122-48b5-83bf-90f9fe6fd81d None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.261507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85769d49-9122-48b5-83bf-90f9fe6fd81d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:41.262032 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85769d49-9122-48b5-83bf-90f9fe6fd81d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:41.264615 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85769d49-9122-48b5-83bf-90f9fe6fd81d None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:41.277460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 415/848] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:41.278553 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85769d49-9122-48b5-83bf-90f9fe6fd81d None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:41.281569 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a38bae-347b-4969-8f0b-73958996dd53 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:41.334397 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897892b-51d1-421e-bcd9-034a6af9200f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:41.335080 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897892b-51d1-421e-bcd9-034a6af9200f None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.335362 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897892b-51d1-421e-bcd9-034a6af9200f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.335636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897892b-51d1-421e-bcd9-034a6af9200f None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.336386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897892b-51d1-421e-bcd9-034a6af9200f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:41.336699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897892b-51d1-421e-bcd9-034a6af9200f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:41.338263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897892b-51d1-421e-bcd9-034a6af9200f None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:41.366277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 434/849] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Sep 22 14:45:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:41.367803 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897892b-51d1-421e-bcd9-034a6af9200f None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:41.369379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:41.371703 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:41.424253 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85769d49-9122-48b5-83bf-90f9fe6fd81d None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:41.480175 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968b6b0-04cb-49c6-9210-7e91088f887c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:41.482125 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968b6b0-04cb-49c6-9210-7e91088f887c None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.483149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968b6b0-04cb-49c6-9210-7e91088f887c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.483801 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968b6b0-04cb-49c6-9210-7e91088f887c None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.485540 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968b6b0-04cb-49c6-9210-7e91088f887c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:41.486254 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968b6b0-04cb-49c6-9210-7e91088f887c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:41.489577 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968b6b0-04cb-49c6-9210-7e91088f887c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:41.498274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 416/850] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:41 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:41.501247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1119e9a-63e0-4e8e-a312-74548396287f None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.501887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1119e9a-63e0-4e8e-a312-74548396287f None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.502487 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897892b-51d1-421e-bcd9-034a6af9200f None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:41.502753 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1119e9a-63e0-4e8e-a312-74548396287f None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.505476 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 417/851] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:41.511176 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6387ffd-fa5f-4b54-a2a8-0db7412d541f None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.511455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6387ffd-fa5f-4b54-a2a8-0db7412d541f None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.511727 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6387ffd-fa5f-4b54-a2a8-0db7412d541f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.534969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-e6387ffd-fa5f-4b54-a2a8-0db7412d541f None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:41.536333 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e6387ffd-fa5f-4b54-a2a8-0db7412d541f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:41.553030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64369cd-7c1d-42d8-8562-8ea92514bbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:41.553955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64369cd-7c1d-42d8-8562-8ea92514bbdd None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.554380 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64369cd-7c1d-42d8-8562-8ea92514bbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.554841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64369cd-7c1d-42d8-8562-8ea92514bbdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.623537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 418/852] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:41.641349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64369cd-7c1d-42d8-8562-8ea92514bbdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:41.643096 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64369cd-7c1d-42d8-8562-8ea92514bbdd None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:41.645521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:41.755824 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 435/853] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:41 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:45:41.761618 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6387ffd-fa5f-4b54-a2a8-0db7412d541f None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:41.832235 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:41.891504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6387ffd-fa5f-4b54-a2a8-0db7412d541f None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:41.914069 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64369cd-7c1d-42d8-8562-8ea92514bbdd None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:41.933189 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b8130-db59-49bf-8f69-cd4c815d87ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:41.933829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b8130-db59-49bf-8f69-cd4c815d87ae None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:41.934109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b8130-db59-49bf-8f69-cd4c815d87ae None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:41.934377 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b8130-db59-49bf-8f69-cd4c815d87ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:41.947541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b8130-db59-49bf-8f69-cd4c815d87ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:41.949036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b8130-db59-49bf-8f69-cd4c815d87ae None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:41.964435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-4e2b8130-db59-49bf-8f69-cd4c815d87ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:41.964435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:41.964435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:41.964435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:41.964435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:41.964435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:41.964435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:41.964435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:41.964435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:41.964435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:41.964435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:41.964435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:41.964435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:41.964435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:41.964435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:41.964435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:41.965458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:41.965458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:41.965458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:41.965458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:41.965458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:41.965458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:41.965458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:41.965458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:41.965458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:41.965458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:41.965458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:41.965458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:41.965458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:41.965458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:41.965458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:41.965458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:41.965458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:41.966347 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:41.966347 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:41.966347 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:41.966347 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 419/854] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:41.972113 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b8130-db59-49bf-8f69-cd4c815d87ae None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:42.044989 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64369cd-7c1d-42d8-8562-8ea92514bbdd None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:42.048357 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:42.091892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ee6c1-1f15-4569-87fa-ca46e8748ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:42.092669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ee6c1-1f15-4569-87fa-ca46e8748ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:42.093016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ee6c1-1f15-4569-87fa-ca46e8748ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:42.093892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021ee6c1-1f15-4569-87fa-ca46e8748ea5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:42.102563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b8130-db59-49bf-8f69-cd4c815d87ae None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:42.104530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021ee6c1-1f15-4569-87fa-ca46e8748ea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:42.105825 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:42.110424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-021ee6c1-1f15-4569-87fa-ca46e8748ea5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:42.110424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:42.110424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:42.110424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:42.110424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:42.110424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:42.110424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:42.110424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:42.110424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:42.110424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:42.110424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:42.110424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:42.110424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:42.110424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:42.110424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:42.110424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:42.111449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:42.111449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:42.111449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:42.111449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:42.111449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:42.111449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:42.111449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:42.111449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:42.111449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:42.111449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:42.111449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:42.111449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:42.111449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:42.111449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:42.111449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:42.111449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:42.112332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:42.112332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:42.112332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:42.112332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:42.112332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:42.112332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:42.112332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:42.112332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:42.112332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:42.112332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:42.112332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:45:42.112332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:42.116555 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 436/855] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:45:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:42.117508 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ee6c1-1f15-4569-87fa-ca46e8748ea5 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:42.154372 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfab29-dc12-4dad-bcd8-5a802d4f4ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:42.155381 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfab29-dc12-4dad-bcd8-5a802d4f4ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:42.155884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfab29-dc12-4dad-bcd8-5a802d4f4ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:42.156328 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfab29-dc12-4dad-bcd8-5a802d4f4ae4 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:42.157159 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dfab29-dc12-4dad-bcd8-5a802d4f4ae4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:42.157690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dfab29-dc12-4dad-bcd8-5a802d4f4ae4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:42.160397 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dfab29-dc12-4dad-bcd8-5a802d4f4ae4 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:42.177379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 420/856] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:42.184211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfab29-dc12-4dad-bcd8-5a802d4f4ae4 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:42.245024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021ee6c1-1f15-4569-87fa-ca46e8748ea5 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:42.292649 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af5fea-b156-442e-89b3-dd6de2800406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:42.294020 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af5fea-b156-442e-89b3-dd6de2800406 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:42.294492 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af5fea-b156-442e-89b3-dd6de2800406 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:42.295297 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af5fea-b156-442e-89b3-dd6de2800406 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:42.296160 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af5fea-b156-442e-89b3-dd6de2800406 None admin] RBAC: Adding query filter params (name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:42.297444 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af5fea-b156-442e-89b3-dd6de2800406 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:42.299870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af5fea-b156-442e-89b3-dd6de2800406 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:42.310371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 437/857] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:45:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:42.318110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af5fea-b156-442e-89b3-dd6de2800406 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:42.330200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfab29-dc12-4dad-bcd8-5a802d4f4ae4 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:42.363153 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:42.370052 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66801672-23b8-4b44-8f0d-0affa6d8c7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:42.371049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66801672-23b8-4b44-8f0d-0affa6d8c7ad None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:42.371711 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66801672-23b8-4b44-8f0d-0affa6d8c7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:42.372254 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66801672-23b8-4b44-8f0d-0affa6d8c7ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:42.380462 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66801672-23b8-4b44-8f0d-0affa6d8c7ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:42.382204 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66801672-23b8-4b44-8f0d-0affa6d8c7ad None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:42.394894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-66801672-23b8-4b44-8f0d-0affa6d8c7ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:42.394894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:42.394894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:42.394894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:42.394894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:42.394894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:42.394894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:42.394894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:42.394894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:42.394894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:42.394894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:42.394894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:42.394894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:42.394894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:42.394894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:42.395869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:42.395869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:42.395869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:42.395869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:42.395869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:42.395869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:42.395869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:42.395869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:42.395869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:42.395869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:42.395869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:42.395869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:42.395869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:42.395869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:42.395869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:42.395869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:42.395869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:42.396784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:42.396784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:42.396784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:42.396784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:42.396784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:42.396784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:42.396784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:42.396784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:42.399291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 421/858] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:42.405733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66801672-23b8-4b44-8f0d-0affa6d8c7ad None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:42.451295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af5fea-b156-442e-89b3-dd6de2800406 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:42.452119 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:42.492931 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a06b7-f3fa-484b-a812-4a0e71c79ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:42.493600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a06b7-f3fa-484b-a812-4a0e71c79ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:42.493881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a06b7-f3fa-484b-a812-4a0e71c79ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:42.494154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a06b7-f3fa-484b-a812-4a0e71c79ba9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:42.507671 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a06b7-f3fa-484b-a812-4a0e71c79ba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:42.509050 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a06b7-f3fa-484b-a812-4a0e71c79ba9 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:42.534265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-e17a06b7-f3fa-484b-a812-4a0e71c79ba9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:42.534265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:42.534265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:42.534265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:42.534265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:42.534265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:42.534265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:42.534265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:42.534265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:42.534265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:42.534265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:42.534265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:42.534265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:42.534265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:42.534265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:42.534265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:42.535353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:42.535353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:42.535353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:42.535353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:42.535353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:42.535353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:42.535353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:42.535353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:42.535353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:42.535353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:42.535353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:42.535353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:42.535353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:42.535353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:42.535353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:42.535353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:42.535353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:42.536276 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:42.536276 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:42.536276 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:42.538678 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 438/859] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:42.545904 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a06b7-f3fa-484b-a812-4a0e71c79ba9 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:42.571041 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66801672-23b8-4b44-8f0d-0affa6d8c7ad None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:42.608590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8587da-80ce-4bbf-973d-9ca059dbf703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:42.609571 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8587da-80ce-4bbf-973d-9ca059dbf703 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:42.610128 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8587da-80ce-4bbf-973d-9ca059dbf703 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:42.610617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8587da-80ce-4bbf-973d-9ca059dbf703 None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:42.611560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8587da-80ce-4bbf-973d-9ca059dbf703 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:42.612064 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8587da-80ce-4bbf-973d-9ca059dbf703 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:42.613696 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8587da-80ce-4bbf-973d-9ca059dbf703 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:42.626852 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 422/860] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:42.636793 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8587da-80ce-4bbf-973d-9ca059dbf703 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:42.663418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:42.691626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a06b7-f3fa-484b-a812-4a0e71c79ba9 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:42.738547 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8316c0-fbd6-4812-9af3-522032566677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:42.739582 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8316c0-fbd6-4812-9af3-522032566677 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:42.740530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8316c0-fbd6-4812-9af3-522032566677 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:42.741038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8316c0-fbd6-4812-9af3-522032566677 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:42.742316 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8316c0-fbd6-4812-9af3-522032566677 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:42.743107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8316c0-fbd6-4812-9af3-522032566677 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:42.744975 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8316c0-fbd6-4812-9af3-522032566677 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:42.748187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:42.765955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 439/861] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:42.771684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a07508-c362-46d0-b42f-39ee8bf074ce None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:42.771961 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a07508-c362-46d0-b42f-39ee8bf074ce None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:42.772232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a07508-c362-46d0-b42f-39ee8bf074ce None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:42.772435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8587da-80ce-4bbf-973d-9ca059dbf703 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:42.773640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 440/862] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:42.774167 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a07508-c362-46d0-b42f-39ee8bf074ce None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:42.811741 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bff9c-ebde-4e3e-92c8-9df32fc28f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:42.812255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bff9c-ebde-4e3e-92c8-9df32fc28f19 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:42.812596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bff9c-ebde-4e3e-92c8-9df32fc28f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:42.812804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bff9c-ebde-4e3e-92c8-9df32fc28f19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:42.813519 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432bff9c-ebde-4e3e-92c8-9df32fc28f19 None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=0a265623ae734936a340c97df852e227) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:42.813828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432bff9c-ebde-4e3e-92c8-9df32fc28f19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:42.815279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432bff9c-ebde-4e3e-92c8-9df32fc28f19 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:42.821204 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 423/863] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:42 2022] GET /identity/v3/role_assignments?user.id=0a265623ae734936a340c97df852e227&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=32c8a81e339a462e973e76bb528ef35a => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:42.822400 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baf9c8-6bbb-489f-b5f4-aacd59ec6d35 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:42.822677 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baf9c8-6bbb-489f-b5f4-aacd59ec6d35 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:42.822976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baf9c8-6bbb-489f-b5f4-aacd59ec6d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:42.843860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-05baf9c8-6bbb-489f-b5f4-aacd59ec6d35 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:42.845107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-05baf9c8-6bbb-489f-b5f4-aacd59ec6d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:42.908640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a07508-c362-46d0-b42f-39ee8bf074ce None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:42.947200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 424/864] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:42.955317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9adf1-7cb9-4f70-b168-2556a665ca48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:42.956023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9adf1-7cb9-4f70-b168-2556a665ca48 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:42.956303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9adf1-7cb9-4f70-b168-2556a665ca48 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:42.956623 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9adf1-7cb9-4f70-b168-2556a665ca48 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:42.960075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:42.966239 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9adf1-7cb9-4f70-b168-2556a665ca48 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:42.967814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9adf1-7cb9-4f70-b168-2556a665ca48 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:42.981895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-f5d9adf1-7cb9-4f70-b168-2556a665ca48 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:42.981895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:42.981895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:42.981895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:42.981895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:42.981895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:42.981895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:42.981895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:42.981895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:42.981895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:42.981895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:42.981895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:42.981895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:42.981895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:42.981895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:42.981895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:42.982938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:42.983914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:42.983914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:42.983914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:42.983914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:42.983914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:42.983914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:42.983914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:42.983914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:42.983914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:42.983914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:42.983914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 441/865] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Sep 22 14:45:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:42.989403 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baf9c8-6bbb-489f-b5f4-aacd59ec6d35 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:43.073814 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:43.097551 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baf9c8-6bbb-489f-b5f4-aacd59ec6d35 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:43.131914 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10775bba-320a-42d7-b640-a92bb46043c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:43.132540 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10775bba-320a-42d7-b640-a92bb46043c1 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:43.132775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10775bba-320a-42d7-b640-a92bb46043c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:43.133043 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10775bba-320a-42d7-b640-a92bb46043c1 None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:43.133764 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10775bba-320a-42d7-b640-a92bb46043c1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:43.134066 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10775bba-320a-42d7-b640-a92bb46043c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:43.135519 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10775bba-320a-42d7-b640-a92bb46043c1 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:43.153456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 425/866] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Sep 22 14:45:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:43.159632 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9adf1-7cb9-4f70-b168-2556a665ca48 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:43.243010 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10775bba-320a-42d7-b640-a92bb46043c1 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:43.269847 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9adf1-7cb9-4f70-b168-2556a665ca48 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:43.304117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c9acb-b6c7-4dd6-8743-38f14a96aaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:43.310702 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c9acb-b6c7-4dd6-8743-38f14a96aaa4 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:43.311021 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c9acb-b6c7-4dd6-8743-38f14a96aaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:43.311299 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c9acb-b6c7-4dd6-8743-38f14a96aaa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:43.372869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10775bba-320a-42d7-b640-a92bb46043c1 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:43.392466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:43.401792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c9acb-b6c7-4dd6-8743-38f14a96aaa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:43.404160 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c9acb-b6c7-4dd6-8743-38f14a96aaa4 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:43.410789 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463461a9-5743-4c14-8015-4e1f7b5a7864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:43.411461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463461a9-5743-4c14-8015-4e1f7b5a7864 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:43.411738 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463461a9-5743-4c14-8015-4e1f7b5a7864 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:43.412007 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463461a9-5743-4c14-8015-4e1f7b5a7864 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:43.412714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463461a9-5743-4c14-8015-4e1f7b5a7864 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:43.414247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463461a9-5743-4c14-8015-4e1f7b5a7864 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:43.418862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:43.433478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 426/867] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Sep 22 14:45:43 2022] POST /identity/v3/services => generated 269 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:45:43.537075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 442/868] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:43 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:45:43.543557 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463461a9-5743-4c14-8015-4e1f7b5a7864 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:43.670437 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463461a9-5743-4c14-8015-4e1f7b5a7864 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:43.708045 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286bd97-9005-457d-90d1-7ef9acce209c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:43.708682 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286bd97-9005-457d-90d1-7ef9acce209c None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:43.708953 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286bd97-9005-457d-90d1-7ef9acce209c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:43.709218 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286bd97-9005-457d-90d1-7ef9acce209c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:43.720676 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286bd97-9005-457d-90d1-7ef9acce209c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:43.722395 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286bd97-9005-457d-90d1-7ef9acce209c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:43.738515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-4286bd97-9005-457d-90d1-7ef9acce209c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:43.738515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:43.738515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:43.738515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:43.738515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:43.738515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:43.738515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:43.738515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:43.738515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:43.738515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:43.738515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:43.738515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:43.738515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:43.738515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:43.738515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:43.738515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:43.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:43.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:43.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:43.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:43.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:43.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:43.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:43.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:43.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:43.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:43.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:43.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:43.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:43.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:43.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:43.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:43.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:43.740941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:43.740941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:43.740941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:43.740941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 427/869] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:43.746225 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c9acb-b6c7-4dd6-8743-38f14a96aaa4 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:43.850073 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:43.856954 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c9acb-b6c7-4dd6-8743-38f14a96aaa4 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:43.862425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0770b3-92b4-46e6-97b9-4cc056548c31 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:43.862703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0770b3-92b4-46e6-97b9-4cc056548c31 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:43.863000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0770b3-92b4-46e6-97b9-4cc056548c31 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:43.864576 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 428/870] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:43.872117 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72150a97-52b3-4248-9fd0-bd8b2e379221 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:43.872529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72150a97-52b3-4248-9fd0-bd8b2e379221 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:43.872894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72150a97-52b3-4248-9fd0-bd8b2e379221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:43.892772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-72150a97-52b3-4248-9fd0-bd8b2e379221 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:43.894155 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-72150a97-52b3-4248-9fd0-bd8b2e379221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:43.908298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b658c-4474-4d19-8b33-5dc95c038dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:43.908922 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b658c-4474-4d19-8b33-5dc95c038dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:43.909202 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b658c-4474-4d19-8b33-5dc95c038dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:43.909473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b658c-4474-4d19-8b33-5dc95c038dc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:43.910125 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b658c-4474-4d19-8b33-5dc95c038dc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:43.910429 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b658c-4474-4d19-8b33-5dc95c038dc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:43.911881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b658c-4474-4d19-8b33-5dc95c038dc5 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:43.925346 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 443/871] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:43.930616 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b658c-4474-4d19-8b33-5dc95c038dc5 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:43.951473 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:43.990901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 429/872] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:43 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:44.043298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b658c-4474-4d19-8b33-5dc95c038dc5 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:44.081014 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ad5ca-979f-4b5c-b368-879c8d20a866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:44.081889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ad5ca-979f-4b5c-b368-879c8d20a866 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:44.082310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ad5ca-979f-4b5c-b368-879c8d20a866 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:44.082771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ad5ca-979f-4b5c-b368-879c8d20a866 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:44.089949 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ad5ca-979f-4b5c-b368-879c8d20a866 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:44.091603 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ad5ca-979f-4b5c-b368-879c8d20a866 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:44.104181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-d19ad5ca-979f-4b5c-b368-879c8d20a866 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:44.104181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:44.104181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:44.104181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:44.104181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:44.104181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:44.104181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:44.104181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:44.104181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:44.104181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:44.104181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:44.104181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:44.104181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:44.104181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:44.104181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:44.105243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:44.105243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:44.105243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:44.105243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:44.105243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:44.105243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:44.105243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:44.105243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:44.105243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:44.105243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:44.105243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:44.105243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:44.105243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:44.105243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:44.105243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:44.105243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:44.105243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:44.106328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:44.106328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:44.106328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:44.106328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:44.106328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:44.106328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:44.106328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:44.106328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:44.108418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 444/873] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:44.115877 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72150a97-52b3-4248-9fd0-bd8b2e379221 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:44.222639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72150a97-52b3-4248-9fd0-bd8b2e379221 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:44.262452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd43218-0a16-4dad-b529-8c80d486c7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:44.263104 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd43218-0a16-4dad-b529-8c80d486c7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:44.263380 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd43218-0a16-4dad-b529-8c80d486c7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:44.263647 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd43218-0a16-4dad-b529-8c80d486c7a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:44.264337 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd43218-0a16-4dad-b529-8c80d486c7a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:44.264641 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd43218-0a16-4dad-b529-8c80d486c7a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:44.266622 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd43218-0a16-4dad-b529-8c80d486c7a0 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:44.279344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 430/874] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:44.285101 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ad5ca-979f-4b5c-b368-879c8d20a866 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:44.285232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd43218-0a16-4dad-b529-8c80d486c7a0 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:44.305461 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:44.330101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:44.404378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ad5ca-979f-4b5c-b368-879c8d20a866 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:44.416955 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd43218-0a16-4dad-b529-8c80d486c7a0 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:44.442159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5007b21-1cd4-42a6-8374-c998e608bf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:44.443125 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5007b21-1cd4-42a6-8374-c998e608bf8e None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:44.443599 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5007b21-1cd4-42a6-8374-c998e608bf8e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:44.444060 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5007b21-1cd4-42a6-8374-c998e608bf8e None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:44.445184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5007b21-1cd4-42a6-8374-c998e608bf8e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:44.446831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5007b21-1cd4-42a6-8374-c998e608bf8e None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:44.457963 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6e4e7-950b-47fa-8c17-74a833531ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:44.458911 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6e4e7-950b-47fa-8c17-74a833531ada None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:44.459194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6e4e7-950b-47fa-8c17-74a833531ada None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:44.459462 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6e4e7-950b-47fa-8c17-74a833531ada None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:44.460211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6e4e7-950b-47fa-8c17-74a833531ada None admin] RBAC: Adding query filter params (role.id=35bdd5122aae45ebbe44158a433d2f99, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=6256407d9df648d5bbe33495cb9a8b36) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:44.460525 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6e4e7-950b-47fa-8c17-74a833531ada None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:44.462529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6e4e7-950b-47fa-8c17-74a833531ada None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:44.472855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 445/876] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Sep 22 14:45:44 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:44.475013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 431/876] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:44 2022] GET /identity/v3/role_assignments?user.id=6256407d9df648d5bbe33495cb9a8b36&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=35bdd5122aae45ebbe44158a433d2f99 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:45.228374 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bcd53-241d-4bbb-8ef4-66e19c28d29c None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:45.228897 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bcd53-241d-4bbb-8ef4-66e19c28d29c None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:45.228973 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bcd53-241d-4bbb-8ef4-66e19c28d29c None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:45.230537 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 446/877] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:45.236264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82a4d9-4ff0-44df-9adf-0ebc393e6e51 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:45.236713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82a4d9-4ff0-44df-9adf-0ebc393e6e51 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:45.236817 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82a4d9-4ff0-44df-9adf-0ebc393e6e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:45.255972 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-2b82a4d9-4ff0-44df-9adf-0ebc393e6e51 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:45.257182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2b82a4d9-4ff0-44df-9adf-0ebc393e6e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:45.350875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 432/878] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:45.619107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bcd53-241d-4bbb-8ef4-66e19c28d29c None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:45.700128 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:45.730627 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bcd53-241d-4bbb-8ef4-66e19c28d29c None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:45.733051 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590129c5-e7b4-495f-841c-07ddb5c25916 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:45.733340 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590129c5-e7b4-495f-841c-07ddb5c25916 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:45.733619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590129c5-e7b4-495f-841c-07ddb5c25916 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:45.736623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 433/879] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:45.741044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e6deb-2945-49cd-97b3-57d48ef42e44 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:45.741335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e6deb-2945-49cd-97b3-57d48ef42e44 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:45.741612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e6deb-2945-49cd-97b3-57d48ef42e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:45.761155 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-619e6deb-2945-49cd-97b3-57d48ef42e44 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:45.762453 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-619e6deb-2945-49cd-97b3-57d48ef42e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:45.769604 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6b838-5fbc-48da-b2be-0be98f5a46ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:45.770239 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6b838-5fbc-48da-b2be-0be98f5a46ba None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:45.770517 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6b838-5fbc-48da-b2be-0be98f5a46ba None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:45.771263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6b838-5fbc-48da-b2be-0be98f5a46ba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:45.771861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be6b838-5fbc-48da-b2be-0be98f5a46ba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:45.773628 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be6b838-5fbc-48da-b2be-0be98f5a46ba None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:45.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-0be6b838-5fbc-48da-b2be-0be98f5a46ba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:45.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:45.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:45.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:45.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:45.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:45.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:45.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:45.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:45.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:45.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:45.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:45.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:45.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:45.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:45.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:45.785510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:45.787761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:45.787761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:45.787761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:45.787761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:45.787761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:45.787761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:45.787761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:45.787761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:45.787761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:45.787761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:45.787761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:45.787761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:45.787761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:45.787761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:45.787761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:45.787761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:45.789426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:45.789426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:45.789426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:45.789426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:45.789426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:45.789426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:45.789426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:45.789426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:45.789426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:45.789426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:45.789426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:45.789426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:45.789426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:45.789426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:45.789426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 447/880] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:45.790845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:45.795588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6b838-5fbc-48da-b2be-0be98f5a46ba None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:45.848392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 434/881] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:45.911740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6b838-5fbc-48da-b2be-0be98f5a46ba None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:45.948239 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60255d83-3a82-401d-918d-b6db51fdd85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:45.949371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60255d83-3a82-401d-918d-b6db51fdd85d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:45.949851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60255d83-3a82-401d-918d-b6db51fdd85d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:45.950301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60255d83-3a82-401d-918d-b6db51fdd85d None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:45.951294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60255d83-3a82-401d-918d-b6db51fdd85d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:45.951932 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60255d83-3a82-401d-918d-b6db51fdd85d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:45.954048 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60255d83-3a82-401d-918d-b6db51fdd85d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:45.964009 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 448/882] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Sep 22 14:45:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:45.965566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e6deb-2945-49cd-97b3-57d48ef42e44 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:46.070433 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e6deb-2945-49cd-97b3-57d48ef42e44 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:46.106546 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49e744-76fa-484e-acf1-8a2ec3488076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:46.107515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49e744-76fa-484e-acf1-8a2ec3488076 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.107996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49e744-76fa-484e-acf1-8a2ec3488076 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.108459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d49e744-76fa-484e-acf1-8a2ec3488076 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.109309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d49e744-76fa-484e-acf1-8a2ec3488076 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:46.109793 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d49e744-76fa-484e-acf1-8a2ec3488076 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:46.111378 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d49e744-76fa-484e-acf1-8a2ec3488076 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:46.120429 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 435/883] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Sep 22 14:45:45 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:46.134951 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60255d83-3a82-401d-918d-b6db51fdd85d None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:46.139589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:46.254325 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60255d83-3a82-401d-918d-b6db51fdd85d None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:46.294833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2a99e-dfc9-417f-a794-6533cc87d1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:46.295518 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2a99e-dfc9-417f-a794-6533cc87d1fd None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.295788 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2a99e-dfc9-417f-a794-6533cc87d1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.296060 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2a99e-dfc9-417f-a794-6533cc87d1fd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.296681 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2a99e-dfc9-417f-a794-6533cc87d1fd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:46.298051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2a99e-dfc9-417f-a794-6533cc87d1fd None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:46.310545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-d9b2a99e-dfc9-417f-a794-6533cc87d1fd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:46.310545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:46.310545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:46.310545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:46.310545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:46.310545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:46.310545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:46.310545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:46.310545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:46.310545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:46.310545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:46.310545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:46.310545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:46.310545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:46.310545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:46.310545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:46.311606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:46.311606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:46.311606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:46.311606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:46.311606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:46.311606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:46.311606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:46.311606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:46.311606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:46.311606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:46.311606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:46.311606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:46.311606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:46.311606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:46.311606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:46.311606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:46.311606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:46.313271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:46.313271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:46.313271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:46.313271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:46.313271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:46.313271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:46.313271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:46.313271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:46.313271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:46.313271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:46.313271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:46.313271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:46.313271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:46.313271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:46.313271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 449/884] 10.222.0.26 () {56 vars in 1132 bytes} [Thu Sep 22 14:45:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:46.317984 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49e744-76fa-484e-acf1-8a2ec3488076 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:46.346322 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:46.440127 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d49e744-76fa-484e-acf1-8a2ec3488076 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:46.477221 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f17e3-001c-40ff-8e73-b50ef0ed9f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:46.477845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f17e3-001c-40ff-8e73-b50ef0ed9f82 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.478120 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f17e3-001c-40ff-8e73-b50ef0ed9f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.478390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f17e3-001c-40ff-8e73-b50ef0ed9f82 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.479055 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f17e3-001c-40ff-8e73-b50ef0ed9f82 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:46.479355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f17e3-001c-40ff-8e73-b50ef0ed9f82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:46.480734 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f17e3-001c-40ff-8e73-b50ef0ed9f82 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:46.486226 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 436/885] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:46.491844 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2a99e-dfc9-417f-a794-6533cc87d1fd None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:46.563225 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:46.599928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2a99e-dfc9-417f-a794-6533cc87d1fd None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:46.635342 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9822285-57d3-4563-b41f-8b5d9ef6b376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:46.635975 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9822285-57d3-4563-b41f-8b5d9ef6b376 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.636255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9822285-57d3-4563-b41f-8b5d9ef6b376 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.636528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9822285-57d3-4563-b41f-8b5d9ef6b376 None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.637179 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9822285-57d3-4563-b41f-8b5d9ef6b376 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:46.637480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9822285-57d3-4563-b41f-8b5d9ef6b376 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:46.638928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9822285-57d3-4563-b41f-8b5d9ef6b376 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:46.645361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 450/886] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:46.651456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f17e3-001c-40ff-8e73-b50ef0ed9f82 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:46.745618 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:46.765843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6af92-b8a9-4fd0-a2f4-17b5b9b4ca81 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.766158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6af92-b8a9-4fd0-a2f4-17b5b9b4ca81 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.766375 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6af92-b8a9-4fd0-a2f4-17b5b9b4ca81 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.767991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 451/887] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:46.768292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f17e3-001c-40ff-8e73-b50ef0ed9f82 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:46.775116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e602ed-6d2a-4687-9585-aed62ecc4c5e None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.775400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e602ed-6d2a-4687-9585-aed62ecc4c5e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.775677 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e602ed-6d2a-4687-9585-aed62ecc4c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.795718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-98e602ed-6d2a-4687-9585-aed62ecc4c5e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:46.797570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-98e602ed-6d2a-4687-9585-aed62ecc4c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:46.814799 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb12d25-aed0-4e93-9042-143bd2daafe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:46.815927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb12d25-aed0-4e93-9042-143bd2daafe7 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.816214 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb12d25-aed0-4e93-9042-143bd2daafe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.816486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb12d25-aed0-4e93-9042-143bd2daafe7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.817526 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb12d25-aed0-4e93-9042-143bd2daafe7 None admin] RBAC: Adding query filter params (interface=public, service_id=aa22e412302a4ba0bd7624a843f581f9, region_id=RegionOne) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:46.817837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb12d25-aed0-4e93-9042-143bd2daafe7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:46.819239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb12d25-aed0-4e93-9042-143bd2daafe7 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:46.825335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 437/888] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Sep 22 14:45:46 2022] GET /identity/v3/endpoints?service_id=aa22e412302a4ba0bd7624a843f581f9&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:46.826397 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d1060-ef12-442c-b2a4-dbd096b61c0e None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.826689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d1060-ef12-442c-b2a4-dbd096b61c0e None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.826971 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d1060-ef12-442c-b2a4-dbd096b61c0e None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.828823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 438/889] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:46.834257 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d1060-ef12-442c-b2a4-dbd096b61c0e None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:46.882785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 452/890] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:46.882785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ecf12-90e6-45f5-b42a-6733209d77c2 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.882785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ecf12-90e6-45f5-b42a-6733209d77c2 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.882785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ecf12-90e6-45f5-b42a-6733209d77c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.906791 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-057ecf12-90e6-45f5-b42a-6733209d77c2 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:46.909291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-057ecf12-90e6-45f5-b42a-6733209d77c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:46.950266 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d1060-ef12-442c-b2a4-dbd096b61c0e None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:46.987395 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:46.994527 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcff32-a47f-45d7-be8c-533fe91ae53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:46.995169 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcff32-a47f-45d7-be8c-533fe91ae53f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:46.995449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcff32-a47f-45d7-be8c-533fe91ae53f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:46.995717 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcff32-a47f-45d7-be8c-533fe91ae53f None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:46.996321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcff32-a47f-45d7-be8c-533fe91ae53f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:46.997687 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcff32-a47f-45d7-be8c-533fe91ae53f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:47.015255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 453/892] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:47.015480 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 439/892] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Sep 22 14:45:46 2022] GET /identity/v3/services => generated 1268 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:45:47.172389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcff32-a47f-45d7-be8c-533fe91ae53f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:47.185570 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:47.285191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ecf12-90e6-45f5-b42a-6733209d77c2 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:47.286845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcff32-a47f-45d7-be8c-533fe91ae53f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:47.327454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c4bf4-2e28-4b0d-b829-041eb1fe76e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:47.328263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c4bf4-2e28-4b0d-b829-041eb1fe76e9 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.328534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c4bf4-2e28-4b0d-b829-041eb1fe76e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.328799 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c4bf4-2e28-4b0d-b829-041eb1fe76e9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.329379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c4bf4-2e28-4b0d-b829-041eb1fe76e9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:47.330768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c4bf4-2e28-4b0d-b829-041eb1fe76e9 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:47.342659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-9d2c4bf4-2e28-4b0d-b829-041eb1fe76e9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:47.342659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:47.342659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:47.342659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:47.342659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:47.342659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:47.342659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:47.342659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:47.342659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:47.342659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:47.342659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:47.342659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:47.342659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:47.342659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:47.342659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:47.342659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:47.342659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:47.344500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:47.344500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:47.344500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:47.344500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:47.344500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:47.344500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:47.344500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:47.344500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:47.344500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:47.344500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:47.344500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:47.344500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:47.344500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:47.344500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:47.344500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:47.344500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:47.345804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:47.345804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:47.345804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:47.345804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:47.345804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:47.345804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:47.345804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:47.345804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:47.345804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:47.345804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:47.345804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:47.345804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:47.345804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:47.345804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:47.345804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 440/893] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:47.350206 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c4bf4-2e28-4b0d-b829-041eb1fe76e9 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:47.412835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ecf12-90e6-45f5-b42a-6733209d77c2 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:47.455242 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69c77b-7db8-4150-9d8c-17075e65fa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:47.456007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69c77b-7db8-4150-9d8c-17075e65fa05 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.456292 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69c77b-7db8-4150-9d8c-17075e65fa05 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.456566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69c77b-7db8-4150-9d8c-17075e65fa05 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.458925 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:47.474767 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69c77b-7db8-4150-9d8c-17075e65fa05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:47.481958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-ab69c77b-7db8-4150-9d8c-17075e65fa05 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:45:47.481958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:47.481958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:47.481958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:47.481958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:47.481958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:47.481958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:47.481958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:47.481958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:47.481958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:47.481958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:47.481958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:47.481958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:47.481958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:47.481958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:47.481958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:47.482939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:47.482939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:47.482939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:47.482939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:47.482939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:47.482939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:47.482939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:47.482939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:47.482939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:47.482939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:47.482939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:47.482939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:47.482939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:47.482939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:47.482939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:47.482939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:47.483866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:47.483866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:47.483866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:47.483866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:47.483866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:47.483866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:47.483866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:47.483866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:47.483866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:47.483866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:47.483866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:45:47.483866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:47.487515 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 454/894] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Sep 22 14:45:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:47.492758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69c77b-7db8-4150-9d8c-17075e65fa05 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:47.498275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c4bf4-2e28-4b0d-b829-041eb1fe76e9 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:47.545757 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cb598-1a99-4b07-83db-aeab7c08ce24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:47.547315 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cb598-1a99-4b07-83db-aeab7c08ce24 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.547820 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cb598-1a99-4b07-83db-aeab7c08ce24 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.548334 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89cb598-1a99-4b07-83db-aeab7c08ce24 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.548965 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:47.549385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cb598-1a99-4b07-83db-aeab7c08ce24 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:47.550427 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cb598-1a99-4b07-83db-aeab7c08ce24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:47.552588 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89cb598-1a99-4b07-83db-aeab7c08ce24 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:47.564349 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 441/895] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:47.566417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cb598-1a99-4b07-83db-aeab7c08ce24 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:47.625317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69c77b-7db8-4150-9d8c-17075e65fa05 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:47.669785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59063681-02fb-49ff-a623-bc00034533a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:47.670389 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59063681-02fb-49ff-a623-bc00034533a4 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.670662 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59063681-02fb-49ff-a623-bc00034533a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.670958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59063681-02fb-49ff-a623-bc00034533a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.671681 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59063681-02fb-49ff-a623-bc00034533a4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:47.671978 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59063681-02fb-49ff-a623-bc00034533a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:47.673395 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59063681-02fb-49ff-a623-bc00034533a4 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:47.686784 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 455/896] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Sep 22 14:45:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:47.687761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59063681-02fb-49ff-a623-bc00034533a4 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:47.689053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89cb598-1a99-4b07-83db-aeab7c08ce24 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:47.734104 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b86440-2797-4e00-95a3-cc094108cd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:47.734631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b86440-2797-4e00-95a3-cc094108cd27 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.734959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b86440-2797-4e00-95a3-cc094108cd27 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.735224 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b86440-2797-4e00-95a3-cc094108cd27 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.735851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b86440-2797-4e00-95a3-cc094108cd27 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:47.736153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b86440-2797-4e00-95a3-cc094108cd27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:47.737798 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b86440-2797-4e00-95a3-cc094108cd27 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:47.747025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 442/897] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:47.749568 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b86440-2797-4e00-95a3-cc094108cd27 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:47.752821 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:47.815170 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59063681-02fb-49ff-a623-bc00034533a4 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:47.838591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:47.865743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b86440-2797-4e00-95a3-cc094108cd27 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:47.866522 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7854e-e20c-438a-908a-7f79a12e2846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:47.867272 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7854e-e20c-438a-908a-7f79a12e2846 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.867626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7854e-e20c-438a-908a-7f79a12e2846 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.868050 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7854e-e20c-438a-908a-7f79a12e2846 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.884197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7854e-e20c-438a-908a-7f79a12e2846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:47.886069 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc7854e-e20c-438a-908a-7f79a12e2846 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:47.904781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-1dc7854e-e20c-438a-908a-7f79a12e2846 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:47.904781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:47.904781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:47.904781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:47.904781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:47.904781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:47.904781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:47.904781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:47.904781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:47.904781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:47.904781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:47.904781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:47.904781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:47.904781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:47.904781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:47.904781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:47.906419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:47.906419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:47.906419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:47.906419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:47.906419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:47.906419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:47.906419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:47.906419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:47.906419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:47.906419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:47.906419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:47.906419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:47.906419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:47.906419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:47.906419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:47.906419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:47.906419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:47.907419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:47.907419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:47.907419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:47.913909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf2c8e-0a4e-4e2c-9ed9-dea43be78a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:47.913909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf2c8e-0a4e-4e2c-9ed9-dea43be78a36 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:47.913909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf2c8e-0a4e-4e2c-9ed9-dea43be78a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:47.913909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf2c8e-0a4e-4e2c-9ed9-dea43be78a36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:47.914286 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf2c8e-0a4e-4e2c-9ed9-dea43be78a36 None admin] RBAC: Adding query filter params (interface=public, service_id=886582ecbad74575b5e1919b17bbd477, region_id=RegionOne) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:47.914286 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf2c8e-0a4e-4e2c-9ed9-dea43be78a36 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:47.914584 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 456/898] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:47.915786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf2c8e-0a4e-4e2c-9ed9-dea43be78a36 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:47.917426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7854e-e20c-438a-908a-7f79a12e2846 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:47.924889 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 443/899] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Sep 22 14:45:47 2022] GET /identity/v3/endpoints?service_id=886582ecbad74575b5e1919b17bbd477&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:47.931679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf2c8e-0a4e-4e2c-9ed9-dea43be78a36 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:48.020533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:48.048506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7854e-e20c-438a-908a-7f79a12e2846 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:48.066604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf2c8e-0a4e-4e2c-9ed9-dea43be78a36 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:48.091174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d989d4-0e90-480d-b4e3-db72e03d19db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:48.092125 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d989d4-0e90-480d-b4e3-db72e03d19db None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:48.092616 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d989d4-0e90-480d-b4e3-db72e03d19db None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:48.093639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d989d4-0e90-480d-b4e3-db72e03d19db None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:48.094781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d989d4-0e90-480d-b4e3-db72e03d19db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:48.095287 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d989d4-0e90-480d-b4e3-db72e03d19db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:48.097963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d989d4-0e90-480d-b4e3-db72e03d19db None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:48.108514 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76b7a0-0688-4fca-a386-cca8ccee2dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:48.109237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76b7a0-0688-4fca-a386-cca8ccee2dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:48.109579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76b7a0-0688-4fca-a386-cca8ccee2dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:48.109912 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76b7a0-0688-4fca-a386-cca8ccee2dc6 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:48.110573 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76b7a0-0688-4fca-a386-cca8ccee2dc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:48.112158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76b7a0-0688-4fca-a386-cca8ccee2dc6 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:48.115841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 457/900] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:48.121749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 444/901] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Sep 22 14:45:47 2022] GET /identity/v3/services => generated 1268 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:45:48.122955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d989d4-0e90-480d-b4e3-db72e03d19db None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:48.164005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:48.250562 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d989d4-0e90-480d-b4e3-db72e03d19db None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:48.293977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658000b-5041-40da-b359-3a8afe0ebaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:48.295704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658000b-5041-40da-b359-3a8afe0ebaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:48.296828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658000b-5041-40da-b359-3a8afe0ebaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:48.297360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658000b-5041-40da-b359-3a8afe0ebaa7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:48.307131 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658000b-5041-40da-b359-3a8afe0ebaa7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:48.310539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658000b-5041-40da-b359-3a8afe0ebaa7 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:48.332343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-6658000b-5041-40da-b359-3a8afe0ebaa7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:48.332343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:48.332343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:48.332343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:48.332343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:48.332343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:48.332343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:48.332343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:48.332343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:48.332343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:48.332343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:48.332343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:48.332343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:48.332343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:48.332343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:48.332343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:48.333354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:48.334657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:48.334657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:48.334657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:48.334657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:48.334657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:48.334657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:48.334657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:48.334657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:48.334657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:48.334657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:48.334657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 458/902] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Sep 22 14:45:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:48.339141 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76b7a0-0688-4fca-a386-cca8ccee2dc6 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:48.420157 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb785421-851a-4194-9898-d68ea6f2dffa None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:48.420497 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb785421-851a-4194-9898-d68ea6f2dffa None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:48.420700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb785421-851a-4194-9898-d68ea6f2dffa None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:48.422290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 459/903] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:48.428258 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c2c6c-65dc-4fcf-ac2e-f62ac7797415 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:48.428546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c2c6c-65dc-4fcf-ac2e-f62ac7797415 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:48.428798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c2c6c-65dc-4fcf-ac2e-f62ac7797415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:48.453318 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-346c2c6c-65dc-4fcf-ac2e-f62ac7797415 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:48.454593 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-346c2c6c-65dc-4fcf-ac2e-f62ac7797415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:48.462345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76b7a0-0688-4fca-a386-cca8ccee2dc6 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:48.496753 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:48.503892 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02987347-3f89-410a-8567-6413cc1f526f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:48.504924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02987347-3f89-410a-8567-6413cc1f526f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:48.505401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02987347-3f89-410a-8567-6413cc1f526f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:48.505889 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02987347-3f89-410a-8567-6413cc1f526f None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:48.506942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02987347-3f89-410a-8567-6413cc1f526f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:48.507553 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02987347-3f89-410a-8567-6413cc1f526f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:48.509318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02987347-3f89-410a-8567-6413cc1f526f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:48.534762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 460/905] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:48.534955 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 445/905] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Sep 22 14:45:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:48.536348 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c2c6c-65dc-4fcf-ac2e-f62ac7797415 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:48.575092 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:48.643274 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c2c6c-65dc-4fcf-ac2e-f62ac7797415 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:48.678647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3824dd-c132-496d-9fcf-e913c6741d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:48.679666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3824dd-c132-496d-9fcf-e913c6741d40 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:48.680183 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3824dd-c132-496d-9fcf-e913c6741d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:48.680654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3824dd-c132-496d-9fcf-e913c6741d40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:48.759949 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3824dd-c132-496d-9fcf-e913c6741d40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:48.761865 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3824dd-c132-496d-9fcf-e913c6741d40 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:48.782707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:48.817589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02987347-3f89-410a-8567-6413cc1f526f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:48.894802 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 461/906] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:48 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:45:48.903334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3824dd-c132-496d-9fcf-e913c6741d40 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:48.998021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02987347-3f89-410a-8567-6413cc1f526f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:49.043987 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560db0d7-f0c1-4856-8b82-096f1777ac50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:49.045382 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560db0d7-f0c1-4856-8b82-096f1777ac50 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:49.045870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560db0d7-f0c1-4856-8b82-096f1777ac50 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:49.047261 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560db0d7-f0c1-4856-8b82-096f1777ac50 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:49.048117 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560db0d7-f0c1-4856-8b82-096f1777ac50 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:49.050600 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560db0d7-f0c1-4856-8b82-096f1777ac50 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:49.056584 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3824dd-c132-496d-9fcf-e913c6741d40 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:49.078215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 446/907] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Sep 22 14:45:48 2022] POST /identity/v3/services => generated 241 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:49.101463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:49.113790 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a48de4-3167-45d3-b53d-4541f98e4828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:49.114436 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a48de4-3167-45d3-b53d-4541f98e4828 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:49.114711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a48de4-3167-45d3-b53d-4541f98e4828 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:49.115012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a48de4-3167-45d3-b53d-4541f98e4828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:49.127742 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a48de4-3167-45d3-b53d-4541f98e4828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:49.129205 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a48de4-3167-45d3-b53d-4541f98e4828 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:49.147025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-21a48de4-3167-45d3-b53d-4541f98e4828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:49.147025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:49.147025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:49.147025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:49.147025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:49.147025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:49.147025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:49.147025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:49.147025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:49.147025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:49.147025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:49.147025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:49.147025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:49.147025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:49.147025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:49.147025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:49.148032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:49.148032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:49.148032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:49.148032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:49.148032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:49.148032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:49.148032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:49.148032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:49.148032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:49.148032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:49.148032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:49.148032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:49.148032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:49.148032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:49.148032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:49.148032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:49.148032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:49.148943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:49.148943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:49.148943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:49.148943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 462/908] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:49.154532 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560db0d7-f0c1-4856-8b82-096f1777ac50 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:49.224012 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:49.285425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560db0d7-f0c1-4856-8b82-096f1777ac50 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:49.316466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79ffdf-602a-4706-80d9-c8fb6d3525ae None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:49.317003 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79ffdf-602a-4706-80d9-c8fb6d3525ae None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:49.317003 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79ffdf-602a-4706-80d9-c8fb6d3525ae None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:49.318322 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 463/909] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:49.323014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ae205-de39-4916-bf1b-b2dce1750150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:49.323861 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ae205-de39-4916-bf1b-b2dce1750150 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:49.324126 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ae205-de39-4916-bf1b-b2dce1750150 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:49.324396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ae205-de39-4916-bf1b-b2dce1750150 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:49.325053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ae205-de39-4916-bf1b-b2dce1750150 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:49.325338 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ae205-de39-4916-bf1b-b2dce1750150 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:49.326839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ae205-de39-4916-bf1b-b2dce1750150 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:49.328644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98fb08-30ca-4b19-bd7f-6d58c05dfe9b None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:49.328932 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98fb08-30ca-4b19-bd7f-6d58c05dfe9b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:49.329206 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98fb08-30ca-4b19-bd7f-6d58c05dfe9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:49.341792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 447/910] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:49.348713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ae205-de39-4916-bf1b-b2dce1750150 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:49.352030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-dd98fb08-30ca-4b19-bd7f-6d58c05dfe9b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:49.353431 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-dd98fb08-30ca-4b19-bd7f-6d58c05dfe9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:49.434539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 464/911] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:49.464869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ae205-de39-4916-bf1b-b2dce1750150 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:49.475209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:49.504222 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad04f10-181d-4d47-8822-279e3df7f44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:49.504867 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad04f10-181d-4d47-8822-279e3df7f44e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:49.505142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad04f10-181d-4d47-8822-279e3df7f44e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:49.505408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad04f10-181d-4d47-8822-279e3df7f44e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:49.512725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad04f10-181d-4d47-8822-279e3df7f44e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:49.514183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad04f10-181d-4d47-8822-279e3df7f44e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:49.526661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-8ad04f10-181d-4d47-8822-279e3df7f44e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:49.526661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:49.526661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:49.526661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:49.526661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:49.526661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:49.526661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:49.526661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:49.526661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:49.526661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:49.526661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:49.526661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:49.526661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:49.526661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:49.526661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:49.527619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:49.527619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:49.527619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:49.527619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:49.527619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:49.527619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:49.527619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:49.527619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:49.527619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:49.527619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:49.527619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:49.527619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:49.527619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:49.527619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:49.527619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:49.527619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:49.527619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:49.528500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:49.528500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:49.528500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:49.528500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:49.528500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:49.528500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:49.528500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:49.528500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:49.528500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 448/912] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:49.536235 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98fb08-30ca-4b19-bd7f-6d58c05dfe9b None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:49.638483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:49.647154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98fb08-30ca-4b19-bd7f-6d58c05dfe9b None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:49.684393 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b9372-6eda-4f90-937c-815718e80c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:49.685055 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b9372-6eda-4f90-937c-815718e80c33 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:49.685272 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b9372-6eda-4f90-937c-815718e80c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:49.685535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b9372-6eda-4f90-937c-815718e80c33 None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:49.686224 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b9372-6eda-4f90-937c-815718e80c33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:49.686577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b9372-6eda-4f90-937c-815718e80c33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:49.688132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b9372-6eda-4f90-937c-815718e80c33 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:49.700521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 465/913] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:49.706910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad04f10-181d-4d47-8822-279e3df7f44e None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:49.737859 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b9372-6eda-4f90-937c-815718e80c33 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:49.832151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad04f10-181d-4d47-8822-279e3df7f44e None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:49.869285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fabd174-fcee-409a-93ca-19511afd662c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:49.869889 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabd174-fcee-409a-93ca-19511afd662c None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:49.870162 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabd174-fcee-409a-93ca-19511afd662c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:49.870427 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fabd174-fcee-409a-93ca-19511afd662c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:49.871230 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabd174-fcee-409a-93ca-19511afd662c None admin] RBAC: Adding query filter params (role.id=4314083fe9cb43b3a969c8567ddef4c7, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=6256407d9df648d5bbe33495cb9a8b36) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:49.871535 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabd174-fcee-409a-93ca-19511afd662c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:49.872953 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fabd174-fcee-409a-93ca-19511afd662c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:49.879409 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 449/914] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:49 2022] GET /identity/v3/role_assignments?user.id=6256407d9df648d5bbe33495cb9a8b36&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=4314083fe9cb43b3a969c8567ddef4c7 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:49.883291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b9372-6eda-4f90-937c-815718e80c33 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:49.927284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138efa0-79d4-440a-acb1-2f29b75205f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:49.928278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138efa0-79d4-440a-acb1-2f29b75205f1 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:49.928739 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138efa0-79d4-440a-acb1-2f29b75205f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:49.929232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138efa0-79d4-440a-acb1-2f29b75205f1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:49.930113 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138efa0-79d4-440a-acb1-2f29b75205f1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:49.931840 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9138efa0-79d4-440a-acb1-2f29b75205f1 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:49.943470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-9138efa0-79d4-440a-acb1-2f29b75205f1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:49.943470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:49.943470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:49.943470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:49.943470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:49.943470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:49.943470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:49.943470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:49.943470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:49.943470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:49.943470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:49.943470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:49.943470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:49.943470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:49.943470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:49.943470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:49.944751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:49.944751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:49.944751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:49.944751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:49.944751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:49.944751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:49.944751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:49.944751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:49.944751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:49.944751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:49.944751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:49.944751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:49.944751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:49.944751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:49.944751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:49.944751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:49.944751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:49.945679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:49.945679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:49.945679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:49.945679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:49.945679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:49.945679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:49.945679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:49.945679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:49.945679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:49.945679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:49.945679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:49.945679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:49.945679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 14:45:49.945679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:49.948309 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 466/915] 10.222.0.26 () {56 vars in 1132 bytes} [Thu Sep 22 14:45:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:49.955597 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fabd174-fcee-409a-93ca-19511afd662c None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:49.995955 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:50.079286 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fabd174-fcee-409a-93ca-19511afd662c None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:50.116195 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10854ce5-4369-448b-a943-b998ab7bb59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:50.117101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10854ce5-4369-448b-a943-b998ab7bb59a None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:50.117539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10854ce5-4369-448b-a943-b998ab7bb59a None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:50.117953 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10854ce5-4369-448b-a943-b998ab7bb59a None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:50.118801 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10854ce5-4369-448b-a943-b998ab7bb59a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:50.119279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10854ce5-4369-448b-a943-b998ab7bb59a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:50.120880 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10854ce5-4369-448b-a943-b998ab7bb59a None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:50.127256 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 450/916] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:50.133352 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138efa0-79d4-440a-acb1-2f29b75205f1 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:50.156361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:50.242362 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138efa0-79d4-440a-acb1-2f29b75205f1 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:50.279912 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28616cc8-9d5c-4ea0-8527-586bfeca2544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:50.280931 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28616cc8-9d5c-4ea0-8527-586bfeca2544 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:50.281410 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28616cc8-9d5c-4ea0-8527-586bfeca2544 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:50.281895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28616cc8-9d5c-4ea0-8527-586bfeca2544 None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:50.282723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28616cc8-9d5c-4ea0-8527-586bfeca2544 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:50.283247 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28616cc8-9d5c-4ea0-8527-586bfeca2544 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:50.284805 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28616cc8-9d5c-4ea0-8527-586bfeca2544 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:50.290195 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 467/917] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:50.295985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10854ce5-4369-448b-a943-b998ab7bb59a None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:50.402040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:50.408729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10854ce5-4369-448b-a943-b998ab7bb59a None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:50.445141 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62079ea-f815-48a6-b710-f78377b11a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:50.445810 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62079ea-f815-48a6-b710-f78377b11a07 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:50.446087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62079ea-f815-48a6-b710-f78377b11a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:50.446359 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62079ea-f815-48a6-b710-f78377b11a07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:50.447127 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62079ea-f815-48a6-b710-f78377b11a07 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:50.448500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62079ea-f815-48a6-b710-f78377b11a07 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:50.472098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 451/918] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:45:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:50.655110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab6e4a-4e3a-4f0c-b252-a8aa286a7f55 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:50.656189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab6e4a-4e3a-4f0c-b252-a8aa286a7f55 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:50.656189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab6e4a-4e3a-4f0c-b252-a8aa286a7f55 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:50.657577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 468/919] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:50.663165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfab809-ef50-4df8-a534-1fd29239c215 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:50.663448 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfab809-ef50-4df8-a534-1fd29239c215 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:50.663720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfab809-ef50-4df8-a534-1fd29239c215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:50.684177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-abfab809-ef50-4df8-a534-1fd29239c215 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:50.685371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-abfab809-ef50-4df8-a534-1fd29239c215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:50.761643 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 452/920] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:51.036315 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab6e4a-4e3a-4f0c-b252-a8aa286a7f55 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:51.123367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:51.144377 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab6e4a-4e3a-4f0c-b252-a8aa286a7f55 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:51.179968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db536ad-66c9-444c-ad4d-6eb08d09abd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:51.180606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db536ad-66c9-444c-ad4d-6eb08d09abd0 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:51.180930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db536ad-66c9-444c-ad4d-6eb08d09abd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:51.181206 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db536ad-66c9-444c-ad4d-6eb08d09abd0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:51.181790 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db536ad-66c9-444c-ad4d-6eb08d09abd0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:51.183178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db536ad-66c9-444c-ad4d-6eb08d09abd0 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:51.192903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-0db536ad-66c9-444c-ad4d-6eb08d09abd0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:51.192903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:51.192903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:51.192903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:51.192903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:51.192903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:51.192903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:51.192903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:51.192903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:51.192903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:51.192903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:51.192903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:51.192903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:51.192903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:51.192903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:51.192903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:51.192903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:51.194920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:51.194920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:51.194920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:51.194920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:51.194920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:51.194920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:51.194920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:51.194920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:51.194920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:51.194920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:51.194920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:51.194920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:51.194920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:51.194920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:51.194920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:51.194920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:51.195861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:51.195861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:51.195861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:51.195861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:51.195861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:51.195861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:51.195861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:51.195861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:51.195861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:51.195861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:51.195861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:51.195861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:51.195861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 14:45:51.195861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:51.195861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 469/921] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:51.200292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfab809-ef50-4df8-a534-1fd29239c215 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:51.287199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:51.315333 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfab809-ef50-4df8-a534-1fd29239c215 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:51.355361 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689fb4d-6399-4fe1-bd3a-950249096c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:51.356023 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689fb4d-6399-4fe1-bd3a-950249096c2a None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:51.356306 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689fb4d-6399-4fe1-bd3a-950249096c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:51.356575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689fb4d-6399-4fe1-bd3a-950249096c2a None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:51.357203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689fb4d-6399-4fe1-bd3a-950249096c2a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:51.357504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689fb4d-6399-4fe1-bd3a-950249096c2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:51.358983 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689fb4d-6399-4fe1-bd3a-950249096c2a None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:51.364086 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 453/922] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:51.371840 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db536ad-66c9-444c-ad4d-6eb08d09abd0 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:51.434626 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9c278-20eb-4341-8151-315ff1e36e56 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:51.435200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9c278-20eb-4341-8151-315ff1e36e56 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:51.435200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9c278-20eb-4341-8151-315ff1e36e56 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:51.436626 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 454/923] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:51.443077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42646a-8709-4b80-8fb7-0820cd80cb38 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:51.443372 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42646a-8709-4b80-8fb7-0820cd80cb38 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:51.443635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42646a-8709-4b80-8fb7-0820cd80cb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:51.465543 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-ee42646a-8709-4b80-8fb7-0820cd80cb38 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:51.467262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-ee42646a-8709-4b80-8fb7-0820cd80cb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:51.490868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db536ad-66c9-444c-ad4d-6eb08d09abd0 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:51.533488 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba4cde-af5b-478e-ae6f-fe40cdc4a2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:51.534692 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba4cde-af5b-478e-ae6f-fe40cdc4a2dd None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:51.535033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba4cde-af5b-478e-ae6f-fe40cdc4a2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:51.535306 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba4cde-af5b-478e-ae6f-fe40cdc4a2dd None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:51.536016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba4cde-af5b-478e-ae6f-fe40cdc4a2dd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:51.536322 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba4cde-af5b-478e-ae6f-fe40cdc4a2dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:51.538096 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba4cde-af5b-478e-ae6f-fe40cdc4a2dd None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:51.545404 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 470/924] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Sep 22 14:45:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:51.550979 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba4cde-af5b-478e-ae6f-fe40cdc4a2dd None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:51.552801 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 455/925] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:51.570765 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:51.665817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba4cde-af5b-478e-ae6f-fe40cdc4a2dd None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:51.700979 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531fbb61-8cf3-49fb-a646-12dca1e8e8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:51.701585 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fbb61-8cf3-49fb-a646-12dca1e8e8ee None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:51.701920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fbb61-8cf3-49fb-a646-12dca1e8e8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:51.702190 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fbb61-8cf3-49fb-a646-12dca1e8e8ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:51.703511 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531fbb61-8cf3-49fb-a646-12dca1e8e8ee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:51.704865 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531fbb61-8cf3-49fb-a646-12dca1e8e8ee None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:51.727066 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 471/926] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:45:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:45:51.873624 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42646a-8709-4b80-8fb7-0820cd80cb38 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:51.952103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:51.986599 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42646a-8709-4b80-8fb7-0820cd80cb38 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:52.029796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa80e091-2d71-44ed-b6f0-96e654f71bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:52.031014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80e091-2d71-44ed-b6f0-96e654f71bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.031341 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80e091-2d71-44ed-b6f0-96e654f71bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.031604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa80e091-2d71-44ed-b6f0-96e654f71bb2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.032158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa80e091-2d71-44ed-b6f0-96e654f71bb2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:52.033572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa80e091-2d71-44ed-b6f0-96e654f71bb2 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:52.044099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-aa80e091-2d71-44ed-b6f0-96e654f71bb2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:52.044099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:52.044099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:52.044099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:52.044099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:52.044099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:52.044099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:52.044099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:52.044099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:52.044099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:52.044099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:52.044099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:52.044099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:52.044099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:52.044099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:52.044099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:52.044099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:52.045170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:52.045170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:52.045170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:52.045170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:52.045170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:52.045170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:52.045170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:52.045170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:52.045170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:52.045170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:52.045170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:52.045170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:52.045170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:52.045170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:52.045170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:52.045170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:52.046025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:52.046025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:52.046025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:52.046025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:52.046025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:52.046025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:52.046025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:52.046025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:52.046025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:52.046025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:52.046025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:52.046025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:52.046025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:52.046025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:52.046025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 456/927] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:52.052885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531fbb61-8cf3-49fb-a646-12dca1e8e8ee None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:52.123428 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:52.147673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcaa8c2-b0ea-47c6-b260-4f819deba5ed None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.147972 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcaa8c2-b0ea-47c6-b260-4f819deba5ed None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.148152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcaa8c2-b0ea-47c6-b260-4f819deba5ed None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.150225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 457/928] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:52.156960 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c8c58-81d9-4ce8-b00d-300bbf774b5e None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.157364 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c8c58-81d9-4ce8-b00d-300bbf774b5e None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.158124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c8c58-81d9-4ce8-b00d-300bbf774b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.167689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531fbb61-8cf3-49fb-a646-12dca1e8e8ee None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:52.179367 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-3c3c8c58-81d9-4ce8-b00d-300bbf774b5e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:52.180569 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3c3c8c58-81d9-4ce8-b00d-300bbf774b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:52.206166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df537648-2a47-4a4a-995b-b477fbae623e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:52.206802 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df537648-2a47-4a4a-995b-b477fbae623e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.207140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df537648-2a47-4a4a-995b-b477fbae623e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.207418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df537648-2a47-4a4a-995b-b477fbae623e None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.208045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df537648-2a47-4a4a-995b-b477fbae623e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:52.208344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df537648-2a47-4a4a-995b-b477fbae623e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:52.209821 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df537648-2a47-4a4a-995b-b477fbae623e None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:52.220590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 472/929] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Sep 22 14:45:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:52.221860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df537648-2a47-4a4a-995b-b477fbae623e None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:52.254978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 458/930] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:52.348875 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df537648-2a47-4a4a-995b-b477fbae623e None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:52.378049 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:52.384855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894601c-751e-4a5d-80c0-b9be7db73b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:52.385476 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894601c-751e-4a5d-80c0-b9be7db73b33 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.385752 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894601c-751e-4a5d-80c0-b9be7db73b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.386023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894601c-751e-4a5d-80c0-b9be7db73b33 None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.386665 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894601c-751e-4a5d-80c0-b9be7db73b33 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:52.386994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894601c-751e-4a5d-80c0-b9be7db73b33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:52.388367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894601c-751e-4a5d-80c0-b9be7db73b33 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:52.394795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 473/931] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Sep 22 14:45:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:52.400636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c8c58-81d9-4ce8-b00d-300bbf774b5e None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:52.467618 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:52.511246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c8c58-81d9-4ce8-b00d-300bbf774b5e None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:52.547606 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ef7cb-7f4e-4cd5-a91a-430ad0032144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:52.548226 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ef7cb-7f4e-4cd5-a91a-430ad0032144 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.548503 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ef7cb-7f4e-4cd5-a91a-430ad0032144 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.548827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ef7cb-7f4e-4cd5-a91a-430ad0032144 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.549640 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ef7cb-7f4e-4cd5-a91a-430ad0032144 None admin] RBAC: Adding query filter params (interface=public, service_id=81012ce1e221458f8d3eb92140e69d50, region_id=RegionOne) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:52.549948 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ef7cb-7f4e-4cd5-a91a-430ad0032144 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:52.551346 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ef7cb-7f4e-4cd5-a91a-430ad0032144 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:52.557372 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 459/932] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Sep 22 14:45:52 2022] GET /identity/v3/endpoints?service_id=81012ce1e221458f8d3eb92140e69d50&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:52.563692 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894601c-751e-4a5d-80c0-b9be7db73b33 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:52.593943 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ef7cb-7f4e-4cd5-a91a-430ad0032144 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:52.700668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894601c-751e-4a5d-80c0-b9be7db73b33 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:52.716472 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ef7cb-7f4e-4cd5-a91a-430ad0032144 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:52.742552 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e43fd-241b-4fd3-bf6e-6fba2f233f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:52.744324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e43fd-241b-4fd3-bf6e-6fba2f233f10 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.744909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e43fd-241b-4fd3-bf6e-6fba2f233f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.745336 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e43fd-241b-4fd3-bf6e-6fba2f233f10 None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.746304 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:52.747644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e43fd-241b-4fd3-bf6e-6fba2f233f10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:52.749883 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e43fd-241b-4fd3-bf6e-6fba2f233f10 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:52.758031 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e37d4-85c0-48cb-8d4e-a09ea32525a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:52.758859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e37d4-85c0-48cb-8d4e-a09ea32525a8 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.759234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e37d4-85c0-48cb-8d4e-a09ea32525a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.759737 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e37d4-85c0-48cb-8d4e-a09ea32525a8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.760119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 474/933] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Sep 22 14:45:52 2022] GET /identity/v3/services => generated 1497 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:45:52.766087 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512a3a0-3021-4ac4-915e-5db0d0c95b27 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.766443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512a3a0-3021-4ac4-915e-5db0d0c95b27 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.766718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512a3a0-3021-4ac4-915e-5db0d0c95b27 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.768385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 475/934] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:52.775929 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c8ff6-4cde-4434-ab68-1b60ba814dcf None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.776259 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c8ff6-4cde-4434-ab68-1b60ba814dcf None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.776543 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c8ff6-4cde-4434-ab68-1b60ba814dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.786106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e37d4-85c0-48cb-8d4e-a09ea32525a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:52.791776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-3a4e37d4-85c0-48cb-8d4e-a09ea32525a8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:45:52.791776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:52.791776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:52.791776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:52.791776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:52.791776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:52.791776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:52.791776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:52.791776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:52.791776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:52.791776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:52.791776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:52.791776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:52.791776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:52.791776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:52.791776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:52.792839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:52.792839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:52.792839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:52.792839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:52.792839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:52.792839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:52.792839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:52.792839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:52.792839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:52.792839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:52.792839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:52.792839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:52.792839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:52.792839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:52.792839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:52.792839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:52.793669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:52.793669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:52.793669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:52.793669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:52.793669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:52.793669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:52.793669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:52.793669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:52.793669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:52.793669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:52.793669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:45:52.793669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:52.797186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 460/935] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Sep 22 14:45:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:52.799893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-1f6c8ff6-4cde-4434-ab68-1b60ba814dcf None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:52.801411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-1f6c8ff6-4cde-4434-ab68-1b60ba814dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:52.803296 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e37d4-85c0-48cb-8d4e-a09ea32525a8 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:52.872633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:52.902790 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 476/936] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:52.944612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e37d4-85c0-48cb-8d4e-a09ea32525a8 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:52.981095 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87438f23-f87e-47d8-ab2f-6ff3b3ff3df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:52.981636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87438f23-f87e-47d8-ab2f-6ff3b3ff3df3 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:52.981915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87438f23-f87e-47d8-ab2f-6ff3b3ff3df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:52.982185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87438f23-f87e-47d8-ab2f-6ff3b3ff3df3 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:52.982878 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87438f23-f87e-47d8-ab2f-6ff3b3ff3df3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:52.983183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87438f23-f87e-47d8-ab2f-6ff3b3ff3df3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:52.985177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87438f23-f87e-47d8-ab2f-6ff3b3ff3df3 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:52.994578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 461/937] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Sep 22 14:45:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:53.000881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c8ff6-4cde-4434-ab68-1b60ba814dcf None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:53.111276 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c8ff6-4cde-4434-ab68-1b60ba814dcf None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:53.148343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bfbe1-cee3-4fc3-b4e6-fc9500039453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:53.149285 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bfbe1-cee3-4fc3-b4e6-fc9500039453 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.149741 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bfbe1-cee3-4fc3-b4e6-fc9500039453 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.150184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bfbe1-cee3-4fc3-b4e6-fc9500039453 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.161805 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bfbe1-cee3-4fc3-b4e6-fc9500039453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:53.163502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bfbe1-cee3-4fc3-b4e6-fc9500039453 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:53.172937 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87438f23-f87e-47d8-ab2f-6ff3b3ff3df3 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:53.181379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-e36bfbe1-cee3-4fc3-b4e6-fc9500039453 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:53.181379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:53.181379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:53.181379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:53.181379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:53.181379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:53.181379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:53.181379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:53.181379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:53.181379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:53.181379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:53.181379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:53.181379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:53.181379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:53.181379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:53.181379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:53.182365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:53.182365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:53.182365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:53.182365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:53.182365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:53.182365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:53.182365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:53.182365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:53.182365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:53.182365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:53.182365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:53.182365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:53.182365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:53.182365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:53.182365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:53.182365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:53.182365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:53.183280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:53.183280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:53.183280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:53.183280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 477/938] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:53.191688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bfbe1-cee3-4fc3-b4e6-fc9500039453 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:53.231813 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:53.237770 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:53.290932 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87438f23-f87e-47d8-ab2f-6ff3b3ff3df3 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:53.302942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bfbe1-cee3-4fc3-b4e6-fc9500039453 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:53.327129 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b8db3-3e3f-4c9e-bdb6-41baf79ee26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:53.327936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b8db3-3e3f-4c9e-bdb6-41baf79ee26d None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.328288 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b8db3-3e3f-4c9e-bdb6-41baf79ee26d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.328757 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b8db3-3e3f-4c9e-bdb6-41baf79ee26d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.329246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b8db3-3e3f-4c9e-bdb6-41baf79ee26d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:53.330698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b8db3-3e3f-4c9e-bdb6-41baf79ee26d None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:53.340787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-3b5b8db3-3e3f-4c9e-bdb6-41baf79ee26d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:45:53.340787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:53.340787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:53.340787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:53.340787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:53.340787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:53.340787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:53.340787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:53.340787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:45:53.340787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:53.340787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:53.340787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:53.340787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:53.340787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:53.340787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:53.340787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:53.340787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:53.341836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:53.341836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:53.341836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:53.341836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:53.341836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:53.341836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:53.341836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:53.341836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:53.341836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:53.341836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:53.341836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:53.341836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:53.341836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:53.341836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:53.341836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:53.341836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:53.342703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:53.342703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:53.342703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:53.342703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:53.342703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:53.342703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:53.342703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:53.342703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:53.342703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:53.342703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:53.342703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:53.342703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:53.342703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:45:53.342703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:53.342703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 462/939] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:53.344063 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36bc2c-6b36-4a20-aaee-97fe9f85eec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:53.344063 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36bc2c-6b36-4a20-aaee-97fe9f85eec4 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.344063 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36bc2c-6b36-4a20-aaee-97fe9f85eec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.344271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36bc2c-6b36-4a20-aaee-97fe9f85eec4 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.345011 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36bc2c-6b36-4a20-aaee-97fe9f85eec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:53.345179 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36bc2c-6b36-4a20-aaee-97fe9f85eec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:53.346510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36bc2c-6b36-4a20-aaee-97fe9f85eec4 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:53.348249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b8db3-3e3f-4c9e-bdb6-41baf79ee26d None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:53.369641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 478/940] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:53.371180 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36bc2c-6b36-4a20-aaee-97fe9f85eec4 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:53.475395 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b8db3-3e3f-4c9e-bdb6-41baf79ee26d None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:53.488930 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:53.495130 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:53.498925 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36bc2c-6b36-4a20-aaee-97fe9f85eec4 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:53.512933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652743f-8441-4963-a4e3-f6266228aa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:53.513511 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652743f-8441-4963-a4e3-f6266228aa55 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.513789 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652743f-8441-4963-a4e3-f6266228aa55 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.514057 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652743f-8441-4963-a4e3-f6266228aa55 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.514677 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652743f-8441-4963-a4e3-f6266228aa55 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:53.515676 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652743f-8441-4963-a4e3-f6266228aa55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:53.517010 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652743f-8441-4963-a4e3-f6266228aa55 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:53.522602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 463/941] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:53.527829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652743f-8441-4963-a4e3-f6266228aa55 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:53.540993 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00f0cb-dda6-486a-be6e-5dafad4d1b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:53.542007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00f0cb-dda6-486a-be6e-5dafad4d1b70 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.542429 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00f0cb-dda6-486a-be6e-5dafad4d1b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.542915 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00f0cb-dda6-486a-be6e-5dafad4d1b70 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.551614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00f0cb-dda6-486a-be6e-5dafad4d1b70 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:53.553438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb00f0cb-dda6-486a-be6e-5dafad4d1b70 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:53.567934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-cb00f0cb-dda6-486a-be6e-5dafad4d1b70 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:53.567934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:53.567934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:53.567934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:53.567934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:53.567934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:53.567934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:53.567934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:53.567934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:53.567934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:53.567934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:53.567934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:53.567934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:53.567934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:53.567934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:53.567934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:53.568930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:53.569921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:53.569921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:53.569921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:53.569921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:53.569921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:53.569921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:53.569921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:53.569921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:53.569921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:53.569921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:53.577160 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 479/942] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Sep 22 14:45:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:53.578877 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00f0cb-dda6-486a-be6e-5dafad4d1b70 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:53.640888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652743f-8441-4963-a4e3-f6266228aa55 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:53.679842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91bd36-c5e4-4fe4-979b-232329f23068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:53.680656 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91bd36-c5e4-4fe4-979b-232329f23068 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.681001 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91bd36-c5e4-4fe4-979b-232329f23068 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.681338 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91bd36-c5e4-4fe4-979b-232329f23068 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.682034 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91bd36-c5e4-4fe4-979b-232329f23068 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:53.682418 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91bd36-c5e4-4fe4-979b-232329f23068 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:53.684035 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91bd36-c5e4-4fe4-979b-232329f23068 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:53.690525 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 464/943] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:53 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:53.692255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00f0cb-dda6-486a-be6e-5dafad4d1b70 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:53.734072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ce410-6fd7-47c1-91db-029340528f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:53.735506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ce410-6fd7-47c1-91db-029340528f95 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.735966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ce410-6fd7-47c1-91db-029340528f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.736434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ce410-6fd7-47c1-91db-029340528f95 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.737356 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ce410-6fd7-47c1-91db-029340528f95 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:53.737861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ce410-6fd7-47c1-91db-029340528f95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:53.739579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ce410-6fd7-47c1-91db-029340528f95 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:53.760470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 480/944] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Sep 22 14:45:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:53.767198 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91bd36-c5e4-4fe4-979b-232329f23068 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:53.817087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:53.886050 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91bd36-c5e4-4fe4-979b-232329f23068 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:53.923559 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b052f-acc4-4159-b6b2-3155f35c492a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:53.924189 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b052f-acc4-4159-b6b2-3155f35c492a None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:53.924459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b052f-acc4-4159-b6b2-3155f35c492a None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:53.924723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b052f-acc4-4159-b6b2-3155f35c492a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:53.935782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b052f-acc4-4159-b6b2-3155f35c492a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:53.937863 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b052f-acc4-4159-b6b2-3155f35c492a None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:53.952923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-125b052f-acc4-4159-b6b2-3155f35c492a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:53.952923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:53.952923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:53.952923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:53.952923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:53.952923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:53.952923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:53.952923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:53.952923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:53.952923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:53.952923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:53.952923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:53.952923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:53.952923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:53.952923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:53.952923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:53.953961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:53.953961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:53.953961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:53.953961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:53.953961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:53.953961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:53.953961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:53.953961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:53.953961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:53.953961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:53.953961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:53.953961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:53.953961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:53.953961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:53.953961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:53.953961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:53.953961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:53.954900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:53.954900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:53.954900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:53.954900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 465/945] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:53.961677 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ce410-6fd7-47c1-91db-029340528f95 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:53.988729 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:54.072326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ce410-6fd7-47c1-91db-029340528f95 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:54.079470 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad0b11-b0c7-4403-ae9d-10d56b907312 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:54.079755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad0b11-b0c7-4403-ae9d-10d56b907312 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:54.080008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad0b11-b0c7-4403-ae9d-10d56b907312 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:54.081680 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 466/946] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:54.086987 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd33d52-461e-4504-ad60-79e9f485e7e3 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:54.087274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd33d52-461e-4504-ad60-79e9f485e7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:54.087548 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd33d52-461e-4504-ad60-79e9f485e7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:54.107156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-2cd33d52-461e-4504-ad60-79e9f485e7e3 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:54.109319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2cd33d52-461e-4504-ad60-79e9f485e7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:54.114474 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c3234-e20c-464f-a2d6-c2c3bd41307c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:54.115111 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c3234-e20c-464f-a2d6-c2c3bd41307c None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:54.115392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c3234-e20c-464f-a2d6-c2c3bd41307c None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:54.115666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c3234-e20c-464f-a2d6-c2c3bd41307c None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:54.116313 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c3234-e20c-464f-a2d6-c2c3bd41307c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:54.116618 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c3234-e20c-464f-a2d6-c2c3bd41307c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:54.117979 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c3234-e20c-464f-a2d6-c2c3bd41307c None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:54.134289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 481/947] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:54.144237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c3234-e20c-464f-a2d6-c2c3bd41307c None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:54.197141 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 467/948] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:54.234329 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:54.258140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c3234-e20c-464f-a2d6-c2c3bd41307c None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:54.293419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9d7f6-beaa-4b62-8c3c-f2869503352b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:54.294386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9d7f6-beaa-4b62-8c3c-f2869503352b None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:54.294916 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9d7f6-beaa-4b62-8c3c-f2869503352b None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:54.295367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9d7f6-beaa-4b62-8c3c-f2869503352b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:54.303081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9d7f6-beaa-4b62-8c3c-f2869503352b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:54.304779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9d7f6-beaa-4b62-8c3c-f2869503352b None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:54.317310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-2dc9d7f6-beaa-4b62-8c3c-f2869503352b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:54.317310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:54.317310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:54.317310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:54.317310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:54.317310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:54.317310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:54.317310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:54.317310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:54.317310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:54.317310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:54.317310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:54.317310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:54.317310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:54.317310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:54.318238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:54.318238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:54.318238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:54.318238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:54.318238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:54.318238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:54.318238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:54.318238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:54.318238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:54.318238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:54.318238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:54.318238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:54.318238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:54.318238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:54.318238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:54.318238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:54.318238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:54.319194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:54.319194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:54.319194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:54.319194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:54.319194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:54.319194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:54.319194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:54.319194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:54.321319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 482/949] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:54.328037 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd33d52-461e-4504-ad60-79e9f485e7e3 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:54.353232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:54.437038 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd33d52-461e-4504-ad60-79e9f485e7e3 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:54.476393 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36b603-0cd6-4208-a019-a2da3e6da15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:54.477052 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36b603-0cd6-4208-a019-a2da3e6da15a None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:54.477328 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36b603-0cd6-4208-a019-a2da3e6da15a None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:54.477607 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36b603-0cd6-4208-a019-a2da3e6da15a None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:54.478301 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36b603-0cd6-4208-a019-a2da3e6da15a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:54.478602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36b603-0cd6-4208-a019-a2da3e6da15a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:54.480090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36b603-0cd6-4208-a019-a2da3e6da15a None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:54.483197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9d7f6-beaa-4b62-8c3c-f2869503352b None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:54.493882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 468/950] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:54.500595 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36b603-0cd6-4208-a019-a2da3e6da15a None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:54.609624 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:54.609936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9d7f6-beaa-4b62-8c3c-f2869503352b None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:54.636435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36b603-0cd6-4208-a019-a2da3e6da15a None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:54.652298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92720491-5196-486e-8ad1-ae0c7d78edf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:54.653193 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92720491-5196-486e-8ad1-ae0c7d78edf5 None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:54.653528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92720491-5196-486e-8ad1-ae0c7d78edf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:54.653900 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92720491-5196-486e-8ad1-ae0c7d78edf5 None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:54.654653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92720491-5196-486e-8ad1-ae0c7d78edf5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:54.656859 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92720491-5196-486e-8ad1-ae0c7d78edf5 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:54.659796 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: WARNING keystone.server.flask.common [None req-92720491-5196-486e-8ad1-ae0c7d78edf5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 22 14:45:54.670164 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:54.681343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.sql.core [None req-92720491-5196-486e-8ad1-ae0c7d78edf5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 14:45:54.681343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 22 14:45:54.681343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [parameters: {'id': '6bc2a4d1c96a4c82aa39bdccf2d80265', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 22 14:45:54.681343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81455) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 14:45:54.688647 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c826a-2ff4-46a8-8ea4-d30f0677241c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:54.689733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c826a-2ff4-46a8-8ea4-d30f0677241c None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:54.690213 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c826a-2ff4-46a8-8ea4-d30f0677241c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:54.690763 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c826a-2ff4-46a8-8ea4-d30f0677241c None admin] PATH_INFO: `/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/6256407d9df648d5bbe33495cb9a8b36/roles/4314083fe9cb43b3a969c8567ddef4c7` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:54.693205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c826a-2ff4-46a8-8ea4-d30f0677241c None admin] RBAC: Authorizing `identity:create_grant(project_id=43bc5871e4bf441b874474bcae4e5385, user_id=6256407d9df648d5bbe33495cb9a8b36, role_id=4314083fe9cb43b3a969c8567ddef4c7)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:54.694929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c826a-2ff4-46a8-8ea4-d30f0677241c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:54.699612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-92720491-5196-486e-8ad1-ae0c7d78edf5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6bc2a4d1c96a4c82aa39bdccf2d80265.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6bc2a4d1c96a4c82aa39bdccf2d80265. Sep 22 14:45:54.699612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:54.699612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 14:45:54.699612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 14:45:54.699612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 14:45:54.699612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 14:45:54.699612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 14:45:54.699612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 14:45:54.699612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 14:45:54.699612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application conn.query(q) Sep 22 14:45:54.699612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 14:45:54.699612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 14:45:54.699612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application result.read() Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 22 14:45:54.701103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 22 14:45:54.703178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 14:45:54.703178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application next(self.gen) Sep 22 14:45:54.703178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 14:45:54.703178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application yield resource Sep 22 14:45:54.703178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 14:45:54.703178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application next(self.gen) Sep 22 14:45:54.703178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 14:45:54.703178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 14:45:54.703178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 14:45:54.703178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 14:45:54.703178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 14:45:54.703178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise self.value Sep 22 14:45:54.703178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 14:45:54.703178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 14:45:54.703178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 14:45:54.703178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application session.commit() Sep 22 14:45:54.703178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 14:45:54.704238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 14:45:54.704238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 14:45:54.704238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 14:45:54.704238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 14:45:54.704238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self.session.flush() Sep 22 14:45:54.704238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 14:45:54.704238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 14:45:54.704238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 14:45:54.704238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 14:45:54.704238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 14:45:54.704238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application compat.raise_( Sep 22 14:45:54.704238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 14:45:54.704238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception Sep 22 14:45:54.704238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 14:45:54.704238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 14:45:54.704238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 14:45:54.704238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 14:45:54.705344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 14:45:54.705344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 14:45:54.705344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 14:45:54.705344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 14:45:54.705344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 22 14:45:54.705344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 22 14:45:54.705344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 14:45:54.705344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 14:45:54.705344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 14:45:54.705344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 14:45:54.705344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 14:45:54.705344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 14:45:54.705344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 14:45:54.705344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 14:45:54.705344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 14:45:54.706305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 14:45:54.706305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 14:45:54.706305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception Sep 22 14:45:54.706305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 14:45:54.706305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 14:45:54.706305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 14:45:54.706305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 14:45:54.706305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 14:45:54.706305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 14:45:54.706305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 14:45:54.706305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application conn.query(q) Sep 22 14:45:54.706305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 14:45:54.706305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 14:45:54.706305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 14:45:54.706305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application result.read() Sep 22 14:45:54.706305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 14:45:54.706305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 14:45:54.707320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 14:45:54.707320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 14:45:54.707320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 14:45:54.707320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 14:45:54.707320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 14:45:54.707320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 14:45:54.707320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 14:45:54.707320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 22 14:45:54.707320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [parameters: {'id': '6bc2a4d1c96a4c82aa39bdccf2d80265', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 22 14:45:54.707320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 14:45:54.707320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:54.707320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:54.707320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:54.707320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:54.707320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 22 14:45:54.708284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 22 14:45:54.708284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 14:45:54.708284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 14:45:54.708284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 22 14:45:54.708284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:54.708284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:54.708284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:54.708284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:54.708284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:54.708284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:54.708284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:54.708284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:54.708284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:54.708284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:54.708284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:54.708284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:54.708284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:54.710037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:54.710037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 22 14:45:54.710037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 22 14:45:54.710037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:54.710037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:54.710037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 22 14:45:54.710037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 22 14:45:54.710037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6bc2a4d1c96a4c82aa39bdccf2d80265. Sep 22 14:45:54.710037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:54.710037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 483/951] 10.222.0.26 () {60 vars in 1140 bytes} [Thu Sep 22 14:45:54 2022] POST /identity/v3/projects => generated 262 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:54.711262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92720491-5196-486e-8ad1-ae0c7d78edf5 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:54.722308 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 469/952] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:45:54 2022] PUT /identity/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/6256407d9df648d5bbe33495cb9a8b36/roles/4314083fe9cb43b3a969c8567ddef4c7 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:45:54.840303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92720491-5196-486e-8ad1-ae0c7d78edf5 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:54.882064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94865a59-2c83-42bd-bb77-802c236e1e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:54.883274 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94865a59-2c83-42bd-bb77-802c236e1e2d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:54.883765 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94865a59-2c83-42bd-bb77-802c236e1e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:54.884251 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94865a59-2c83-42bd-bb77-802c236e1e2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:54.893996 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94865a59-2c83-42bd-bb77-802c236e1e2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:54.897215 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94865a59-2c83-42bd-bb77-802c236e1e2d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:54.911674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-94865a59-2c83-42bd-bb77-802c236e1e2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:54.911674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:54.911674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:54.911674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:54.911674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:54.911674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:54.911674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:54.911674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:54.911674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:54.911674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:54.911674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:54.911674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:54.911674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:54.911674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:54.911674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:54.912690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:54.912690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:54.912690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:54.912690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:54.912690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:54.912690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:54.912690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:54.912690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:54.912690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:54.912690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:54.912690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:54.912690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:54.912690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:54.912690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:54.912690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:54.912690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:54.912690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:54.913852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:54.913852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:54.913852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:54.913852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:54.913852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:54.913852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:54.913852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:54.913852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:54.916180 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 484/953] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:54.922038 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c826a-2ff4-46a8-8ea4-d30f0677241c None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:55.043551 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c826a-2ff4-46a8-8ea4-d30f0677241c None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:55.056306 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4fad4-a3c4-4f47-b6b9-50e45c17d435 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:55.056727 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4fad4-a3c4-4f47-b6b9-50e45c17d435 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:55.057105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4fad4-a3c4-4f47-b6b9-50e45c17d435 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:55.062523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 485/954] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:55.069016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0b337-678b-4461-a377-e366fb72bfb0 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:55.069441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0b337-678b-4461-a377-e366fb72bfb0 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:55.069810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0b337-678b-4461-a377-e366fb72bfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:55.089815 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301041f5-6c32-4c5e-a06a-e27cd822cb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:55.090334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-28f0b337-678b-4461-a377-e366fb72bfb0 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:55.090907 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301041f5-6c32-4c5e-a06a-e27cd822cb53 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:55.091322 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301041f5-6c32-4c5e-a06a-e27cd822cb53 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:55.091728 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301041f5-6c32-4c5e-a06a-e27cd822cb53 None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:55.091933 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-28f0b337-678b-4461-a377-e366fb72bfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:55.092720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301041f5-6c32-4c5e-a06a-e27cd822cb53 None admin] RBAC: Adding query filter params (name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:55.093753 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301041f5-6c32-4c5e-a06a-e27cd822cb53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:55.097047 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301041f5-6c32-4c5e-a06a-e27cd822cb53 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:55.120773 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 470/955] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:55.154847 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:55.175647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 486/956] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:55.447379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301041f5-6c32-4c5e-a06a-e27cd822cb53 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:55.450394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:55.562803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301041f5-6c32-4c5e-a06a-e27cd822cb53 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:55.601497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621309eb-9ee0-4fa5-8eb3-217e4551ae0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:55.602191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621309eb-9ee0-4fa5-8eb3-217e4551ae0b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:55.602471 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621309eb-9ee0-4fa5-8eb3-217e4551ae0b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:55.602755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621309eb-9ee0-4fa5-8eb3-217e4551ae0b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:55.603394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621309eb-9ee0-4fa5-8eb3-217e4551ae0b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:55.604775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621309eb-9ee0-4fa5-8eb3-217e4551ae0b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:55.614650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-621309eb-9ee0-4fa5-8eb3-217e4551ae0b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:55.614650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:55.614650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:55.614650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:55.614650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:55.614650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:55.614650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:55.614650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:55.614650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:55.614650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:55.614650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:55.614650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:55.614650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:55.614650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:55.614650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:55.614650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:55.614650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:55.615750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:55.615750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:55.615750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:55.615750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:55.615750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:55.615750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:55.615750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:55.615750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:55.615750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:55.615750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:55.615750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:55.615750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:55.615750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:55.615750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:55.615750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:55.615750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:55.616617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:55.616617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:55.616617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:55.616617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:55.616617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:55.616617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:55.616617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:55.616617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:55.616617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:55.616617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:55.616617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:55.616617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:55.616617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:55.616617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:55.616617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 471/957] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:55.622108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0b337-678b-4461-a377-e366fb72bfb0 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:55.741763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0b337-678b-4461-a377-e366fb72bfb0 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:55.777897 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a81f7-1376-41b6-87c5-f62b0f780014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:55.778664 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a81f7-1376-41b6-87c5-f62b0f780014 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:55.779063 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a81f7-1376-41b6-87c5-f62b0f780014 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:55.779427 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a81f7-1376-41b6-87c5-f62b0f780014 None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:55.780150 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442a81f7-1376-41b6-87c5-f62b0f780014 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:55.780574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442a81f7-1376-41b6-87c5-f62b0f780014 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:55.782013 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442a81f7-1376-41b6-87c5-f62b0f780014 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:55.787434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 487/958] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Sep 22 14:45:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:55.793045 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621309eb-9ee0-4fa5-8eb3-217e4551ae0b None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:55.865397 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:55.911104 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621309eb-9ee0-4fa5-8eb3-217e4551ae0b None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:55.949339 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b9f68-1fdf-4587-8992-689cc4a66377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:55.950062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b9f68-1fdf-4587-8992-689cc4a66377 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:55.950339 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b9f68-1fdf-4587-8992-689cc4a66377 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:55.950609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b9f68-1fdf-4587-8992-689cc4a66377 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:55.951407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b9f68-1fdf-4587-8992-689cc4a66377 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:55.951790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b9f68-1fdf-4587-8992-689cc4a66377 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:55.953361 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b9f68-1fdf-4587-8992-689cc4a66377 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:55.958964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 472/959] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Sep 22 14:45:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:55.964876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a81f7-1376-41b6-87c5-f62b0f780014 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:56.022653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:56.043154 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154d455-2a56-48b9-a54c-1dc48ce56151 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:56.043600 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154d455-2a56-48b9-a54c-1dc48ce56151 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:56.043977 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154d455-2a56-48b9-a54c-1dc48ce56151 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:56.045681 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 473/960] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:56.056255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49c998-403a-45a8-a6d4-3e4c6f69773c None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:56.056625 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49c998-403a-45a8-a6d4-3e4c6f69773c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:56.057008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49c998-403a-45a8-a6d4-3e4c6f69773c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:56.091058 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-9e49c998-403a-45a8-a6d4-3e4c6f69773c None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:56.092894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-9e49c998-403a-45a8-a6d4-3e4c6f69773c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:56.093133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a81f7-1376-41b6-87c5-f62b0f780014 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:56.136093 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a32d73-4d8c-453d-84cd-77c54d46ce77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:56.136967 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a32d73-4d8c-453d-84cd-77c54d46ce77 None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:56.137394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a32d73-4d8c-453d-84cd-77c54d46ce77 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:56.137849 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a32d73-4d8c-453d-84cd-77c54d46ce77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:56.138753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a32d73-4d8c-453d-84cd-77c54d46ce77 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:56.141751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a32d73-4d8c-453d-84cd-77c54d46ce77 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:56.170926 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 488/961] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:45:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:56.208132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 474/962] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:56.495997 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a32d73-4d8c-453d-84cd-77c54d46ce77 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:56.604988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a32d73-4d8c-453d-84cd-77c54d46ce77 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:56.607712 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:56.644797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e9aed4-3295-4772-9ed7-aea4900fc053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:56.645662 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9aed4-3295-4772-9ed7-aea4900fc053 None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:56.646119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9aed4-3295-4772-9ed7-aea4900fc053 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:56.646544 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9aed4-3295-4772-9ed7-aea4900fc053 None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:56.647380 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9aed4-3295-4772-9ed7-aea4900fc053 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:56.648893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e9aed4-3295-4772-9ed7-aea4900fc053 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:56.665878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 489/963] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Sep 22 14:45:56 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:45:57.093678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22253cac-a08a-4a17-9c3e-7ca15db594c0 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:57.094198 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22253cac-a08a-4a17-9c3e-7ca15db594c0 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:57.094198 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22253cac-a08a-4a17-9c3e-7ca15db594c0 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:57.095842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 475/964] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:57.101620 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac872200-e4fb-4a29-8048-e301b15d1ce3 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:57.102132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac872200-e4fb-4a29-8048-e301b15d1ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:57.102211 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac872200-e4fb-4a29-8048-e301b15d1ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:57.122443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-ac872200-e4fb-4a29-8048-e301b15d1ce3 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:57.123711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ac872200-e4fb-4a29-8048-e301b15d1ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:57.203250 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 490/965] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:57.448907 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d0c78-867a-4d92-ab67-10d5a5cc3583 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:57.449087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d0c78-867a-4d92-ab67-10d5a5cc3583 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:57.449364 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d0c78-867a-4d92-ab67-10d5a5cc3583 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:57.451011 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 476/966] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:57.456934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f5893-86a2-4e02-97f6-a14ee1e1bb76 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:57.457223 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f5893-86a2-4e02-97f6-a14ee1e1bb76 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:57.457502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f5893-86a2-4e02-97f6-a14ee1e1bb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:57.477031 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-e57f5893-86a2-4e02-97f6-a14ee1e1bb76 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:57.478233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e57f5893-86a2-4e02-97f6-a14ee1e1bb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:57.497376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d0c78-867a-4d92-ab67-10d5a5cc3583 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:57.559208 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 491/967] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:57.568730 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:57.613645 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d0c78-867a-4d92-ab67-10d5a5cc3583 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:57.650547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd13f7-b7ac-4d40-af33-f759336cd1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:57.651205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd13f7-b7ac-4d40-af33-f759336cd1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:57.651478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd13f7-b7ac-4d40-af33-f759336cd1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:57.651743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd13f7-b7ac-4d40-af33-f759336cd1c7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:57.663063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd13f7-b7ac-4d40-af33-f759336cd1c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:45:57.668025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-59bd13f7-b7ac-4d40-af33-f759336cd1c7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:45:57.668025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:57.668025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:57.668025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:57.668025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:57.668025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:57.668025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:57.668025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:57.668025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:57.668025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:57.668025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:57.668025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:57.668025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:45:57.668025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:45:57.668025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:45:57.668025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:45:57.668980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:45:57.668980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:45:57.668980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:45:57.668980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:45:57.668980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:57.668980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:57.668980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:57.668980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:57.668980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:57.668980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:57.668980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:57.668980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:57.668980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:57.668980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:57.668980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:57.668980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:57.669864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:57.669864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:57.669864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:57.669864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:57.669864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:45:57.669864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:45:57.669864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:45:57.669864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:45:57.669864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:45:57.669864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:45:57.669864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:45:57.669864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:57.669864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 477/968] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Sep 22 14:45:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:57.675405 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f5893-86a2-4e02-97f6-a14ee1e1bb76 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:57.702200 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:57.795549 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f5893-86a2-4e02-97f6-a14ee1e1bb76 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:57.832670 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a88d78-648c-4717-9d1c-30723144bf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:57.833277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a88d78-648c-4717-9d1c-30723144bf04 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:57.833552 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a88d78-648c-4717-9d1c-30723144bf04 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:57.833823 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a88d78-648c-4717-9d1c-30723144bf04 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:57.834479 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a88d78-648c-4717-9d1c-30723144bf04 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:57.834802 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a88d78-648c-4717-9d1c-30723144bf04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:57.836248 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a88d78-648c-4717-9d1c-30723144bf04 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:57.837898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd13f7-b7ac-4d40-af33-f759336cd1c7 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:57.852092 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 492/969] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Sep 22 14:45:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:57.857908 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a88d78-648c-4717-9d1c-30723144bf04 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:57.947020 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd13f7-b7ac-4d40-af33-f759336cd1c7 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:57.955528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:57.964703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:57.979190 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a88d78-648c-4717-9d1c-30723144bf04 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:57.986855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822a303-e0fa-499e-84dc-808b862c5caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:57.987684 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822a303-e0fa-499e-84dc-808b862c5caf None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:57.988112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822a303-e0fa-499e-84dc-808b862c5caf None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:57.988563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822a303-e0fa-499e-84dc-808b862c5caf None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:57.989369 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822a303-e0fa-499e-84dc-808b862c5caf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:57.991065 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822a303-e0fa-499e-84dc-808b862c5caf None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:57.993939 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: WARNING keystone.server.flask.common [None req-5822a303-e0fa-499e-84dc-808b862c5caf None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 22 14:45:58.008616 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.sql.core [None req-5822a303-e0fa-499e-84dc-808b862c5caf None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 14:45:58.008616 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 22 14:45:58.008616 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [parameters: {'user_id': '06bd537520384b9ebe0dc77b498bc38b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 22 14:45:58.008616 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81454) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 14:45:58.017535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f12ee-57ca-4730-8015-e222b3cb08d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:58.018248 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f12ee-57ca-4730-8015-e222b3cb08d1 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.018593 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f12ee-57ca-4730-8015-e222b3cb08d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.018990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f12ee-57ca-4730-8015-e222b3cb08d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.019243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-5822a303-e0fa-499e-84dc-808b862c5caf None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 22 14:45:58.019243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:58.019243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 14:45:58.019243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 14:45:58.019243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 14:45:58.019243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 14:45:58.019243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 14:45:58.019243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 14:45:58.019243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 14:45:58.019243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application conn.query(q) Sep 22 14:45:58.019243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 14:45:58.019243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 14:45:58.019243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 14:45:58.019243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application result.read() Sep 22 14:45:58.019243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 14:45:58.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application next(self.gen) Sep 22 14:45:58.021432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 14:45:58.021432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application yield resource Sep 22 14:45:58.021432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 14:45:58.021432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application next(self.gen) Sep 22 14:45:58.021432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 14:45:58.021432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 14:45:58.021432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 14:45:58.021432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 14:45:58.021432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 14:45:58.021432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise self.value Sep 22 14:45:58.021432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 14:45:58.021432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 14:45:58.021432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 14:45:58.021432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application session.commit() Sep 22 14:45:58.021432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 14:45:58.021432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 14:45:58.021432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 14:45:58.022581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 14:45:58.022581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 14:45:58.022581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self.session.flush() Sep 22 14:45:58.022581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 14:45:58.022581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 14:45:58.022581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 14:45:58.022581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 14:45:58.022581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 14:45:58.022581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application compat.raise_( Sep 22 14:45:58.022581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 14:45:58.022581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception Sep 22 14:45:58.022581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 14:45:58.022581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 14:45:58.022581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 14:45:58.022581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 14:45:58.022581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 14:45:58.022581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 14:45:58.024046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 14:45:58.024046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 14:45:58.024046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 22 14:45:58.024046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 22 14:45:58.024046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 14:45:58.024046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 14:45:58.024046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 14:45:58.024046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 14:45:58.024046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 14:45:58.024046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 14:45:58.024046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 14:45:58.024046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 14:45:58.024046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 14:45:58.024046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 14:45:58.024046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 14:45:58.024046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception Sep 22 14:45:58.025316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 14:45:58.025316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 14:45:58.025316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 14:45:58.025316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 14:45:58.025316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 14:45:58.025316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 14:45:58.025316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 14:45:58.025316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application conn.query(q) Sep 22 14:45:58.025316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 14:45:58.025316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 14:45:58.025316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 14:45:58.025316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application result.read() Sep 22 14:45:58.025316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 14:45:58.025316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 14:45:58.025316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 14:45:58.025316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 14:45:58.026283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 14:45:58.026283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 14:45:58.026283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 14:45:58.026283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 14:45:58.026283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 14:45:58.026283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 22 14:45:58.026283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [parameters: {'user_id': '06bd537520384b9ebe0dc77b498bc38b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 22 14:45:58.026283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 14:45:58.026283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:58.026283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:58.026283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:58.026283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:58.026283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:58.026283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:58.026283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:58.027239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:58.027239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:58.027239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:58.027239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:58.027239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:58.027239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:58.027239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:58.027239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 22 14:45:58.027239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 22 14:45:58.027239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:58.027239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:58.027239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:58.027239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:58.027239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:58.027239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:58.027239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 22 14:45:58.027239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 22 14:45:58.028336 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 22 14:45:58.028336 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 22 14:45:58.028336 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 14:45:58.028336 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 14:45:58.028336 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 22 14:45:58.028336 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:58.036785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 478/970] 10.222.0.26 () {60 vars in 1131 bytes} [Thu Sep 22 14:45:57 2022] POST /identity/v3/users => generated 160 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:58.039006 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f12ee-57ca-4730-8015-e222b3cb08d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:58.040349 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822a303-e0fa-499e-84dc-808b862c5caf None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:58.040684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f12ee-57ca-4730-8015-e222b3cb08d1 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.061116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-ad5f12ee-57ca-4730-8015-e222b3cb08d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:58.061116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:58.061116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:58.061116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:58.061116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:58.061116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:58.061116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:58.061116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:58.061116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:58.061116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:58.061116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:58.061116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:58.061116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:58.061116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:58.061116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:58.061116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:58.062379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:58.062379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:58.062379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:58.062379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:58.062379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:58.062379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:58.062379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:58.062379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:58.062379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:58.062379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:58.062379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:58.062379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:58.062379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:58.062379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:58.062379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:58.062379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:58.062379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:58.063513 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:58.063513 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:58.063513 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:58.066815 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 493/971] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:58.071334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f12ee-57ca-4730-8015-e222b3cb08d1 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:58.180329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822a303-e0fa-499e-84dc-808b862c5caf None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:58.198724 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f12ee-57ca-4730-8015-e222b3cb08d1 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:58.229317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de77366-bb51-4ac3-abf8-b29c51b3b7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:58.230889 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de77366-bb51-4ac3-abf8-b29c51b3b7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.231321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de77366-bb51-4ac3-abf8-b29c51b3b7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.231740 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de77366-bb51-4ac3-abf8-b29c51b3b7b8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.240807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627cd7d-68b6-48e6-9c49-a9f180cb2de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:58.241480 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de77366-bb51-4ac3-abf8-b29c51b3b7b8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:58.241602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627cd7d-68b6-48e6-9c49-a9f180cb2de1 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.241883 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627cd7d-68b6-48e6-9c49-a9f180cb2de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.242222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627cd7d-68b6-48e6-9c49-a9f180cb2de1 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.243036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627cd7d-68b6-48e6-9c49-a9f180cb2de1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:58.243418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627cd7d-68b6-48e6-9c49-a9f180cb2de1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:58.244186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de77366-bb51-4ac3-abf8-b29c51b3b7b8 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.244908 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c627cd7d-68b6-48e6-9c49-a9f180cb2de1 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.246689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:58.260940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-3de77366-bb51-4ac3-abf8-b29c51b3b7b8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 14:45:58.260940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:58.260940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:58.260940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:58.260940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:58.260940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:58.260940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:58.260940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:58.260940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:58.260940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:58.260940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:58.260940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:58.260940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:58.260940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:58.260940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:58.260940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:58.262108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:58.263385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:58.263385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:58.263385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:58.263385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:58.263385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:58.263385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:58.263385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:58.263385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:58.263385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 14:45:58.263385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:58.264056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 494/972] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:58.265921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 479/973] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:45:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:58.271976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de77366-bb51-4ac3-abf8-b29c51b3b7b8 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:58.272116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627cd7d-68b6-48e6-9c49-a9f180cb2de1 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:58.321277 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:58.397811 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de77366-bb51-4ac3-abf8-b29c51b3b7b8 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:58.406228 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627cd7d-68b6-48e6-9c49-a9f180cb2de1 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:58.438700 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9826cf1-f8ad-4e63-80a7-d79d14501d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:58.439716 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9826cf1-f8ad-4e63-80a7-d79d14501d81 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.440194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9826cf1-f8ad-4e63-80a7-d79d14501d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.440673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9826cf1-f8ad-4e63-80a7-d79d14501d81 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.449937 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9826cf1-f8ad-4e63-80a7-d79d14501d81 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:58.451978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9826cf1-f8ad-4e63-80a7-d79d14501d81 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.453169 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170b5c8-e84b-448c-9a9b-494e82e3ced5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:58.453898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170b5c8-e84b-448c-9a9b-494e82e3ced5 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.454279 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170b5c8-e84b-448c-9a9b-494e82e3ced5 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.454648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170b5c8-e84b-448c-9a9b-494e82e3ced5 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.455582 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170b5c8-e84b-448c-9a9b-494e82e3ced5 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:58.455986 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170b5c8-e84b-448c-9a9b-494e82e3ced5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:58.457614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170b5c8-e84b-448c-9a9b-494e82e3ced5 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.466274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-c9826cf1-f8ad-4e63-80a7-d79d14501d81 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:58.466274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:58.466274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:58.466274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:58.466274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:58.466274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:58.466274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:58.466274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:58.466274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:58.466274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:58.466274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:58.466274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:58.466274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:45:58.466274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:45:58.466274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:45:58.466274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:58.467455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:58.469552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:58.469552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:58.469552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:45:58.469552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:45:58.469552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:45:58.469552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:45:58.469552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:45:58.469552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:45:58.469552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 14:45:58.469552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:58.472159 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 480/974] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Sep 22 14:45:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:58.478879 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9826cf1-f8ad-4e63-80a7-d79d14501d81 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:58.506191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 495/975] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Sep 22 14:45:58 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:58.534258 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae4257-5680-462a-90b1-298115d92e67 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.534821 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae4257-5680-462a-90b1-298115d92e67 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.535229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae4257-5680-462a-90b1-298115d92e67 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.537119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 496/976] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:45:58.543105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd22fd-a893-4970-8148-8b60e417600a None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.543619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd22fd-a893-4970-8148-8b60e417600a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.544146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd22fd-a893-4970-8148-8b60e417600a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.563956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-1cdd22fd-a893-4970-8148-8b60e417600a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:58.565423 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-1cdd22fd-a893-4970-8148-8b60e417600a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:58.615177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:58.622286 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9826cf1-f8ad-4e63-80a7-d79d14501d81 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:58.623737 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:58.658806 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 497/977] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:45:58.676960 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c9fc0-e971-4cc9-9aec-84bc6d4b31d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:58.677882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c9fc0-e971-4cc9-9aec-84bc6d4b31d5 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.678313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c9fc0-e971-4cc9-9aec-84bc6d4b31d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.678764 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c9fc0-e971-4cc9-9aec-84bc6d4b31d5 None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.679662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c9fc0-e971-4cc9-9aec-84bc6d4b31d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:58.680159 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c9fc0-e971-4cc9-9aec-84bc6d4b31d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:58.681796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c9fc0-e971-4cc9-9aec-84bc6d4b31d5 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.704537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 481/978] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Sep 22 14:45:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:58.710991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd22fd-a893-4970-8148-8b60e417600a None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:58.823612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd22fd-a893-4970-8148-8b60e417600a None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:58.861206 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284ff4c-96de-4bc8-985b-b6dc697173a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:58.861824 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ff4c-96de-4bc8-985b-b6dc697173a7 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:58.862098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ff4c-96de-4bc8-985b-b6dc697173a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:58.862365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ff4c-96de-4bc8-985b-b6dc697173a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:58.916984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:58.937537 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284ff4c-96de-4bc8-985b-b6dc697173a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:58.939033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284ff4c-96de-4bc8-985b-b6dc697173a7 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:58.941387 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c9fc0-e971-4cc9-9aec-84bc6d4b31d5 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:59.045584 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 498/979] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:45:58 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:45:59.047080 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3a228-f018-434e-ae69-88e7a9a5ca49 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:59.047366 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3a228-f018-434e-ae69-88e7a9a5ca49 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:59.047641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3a228-f018-434e-ae69-88e7a9a5ca49 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:59.048894 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 499/980] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:45:59.051285 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3a228-f018-434e-ae69-88e7a9a5ca49 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:59.056841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c9fc0-e971-4cc9-9aec-84bc6d4b31d5 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:59.091144 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c70403-9862-4c3f-bb70-ee4d2ffa2f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:59.091701 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c70403-9862-4c3f-bb70-ee4d2ffa2f58 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:59.091980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c70403-9862-4c3f-bb70-ee4d2ffa2f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:59.092243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c70403-9862-4c3f-bb70-ee4d2ffa2f58 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:59.093820 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c70403-9862-4c3f-bb70-ee4d2ffa2f58 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:59.095221 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c70403-9862-4c3f-bb70-ee4d2ffa2f58 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:59.104405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-08c70403-9862-4c3f-bb70-ee4d2ffa2f58 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:59.104405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:59.104405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:59.104405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:59.104405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:59.104405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:59.104405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:59.104405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:59.104405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:59.104405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:59.104405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:59.104405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:59.104405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:59.104405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:59.104405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:59.104405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:59.104405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:59.105493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:59.105493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:59.105493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:59.105493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:59.105493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:59.105493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:59.105493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:59.105493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:59.105493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:59.105493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:59.105493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:59.105493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:59.105493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:59.105493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:59.105493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:59.105493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:59.106376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:59.106376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:59.106376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:59.106376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:59.106376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:59.106376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:59.106376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:59.106376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:59.106376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:59.106376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:59.106376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:59.106376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:59.106376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:45:59.106376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:59.106376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 482/981] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Sep 22 14:45:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:59.107252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b95928-c867-4de1-9146-66801c42cb59 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:59.107321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b95928-c867-4de1-9146-66801c42cb59 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:59.107911 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b95928-c867-4de1-9146-66801c42cb59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:59.125006 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-08b95928-c867-4de1-9146-66801c42cb59 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:45:59.126498 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-08b95928-c867-4de1-9146-66801c42cb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:45:59.137372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:59.141113 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:59.158486 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3a228-f018-434e-ae69-88e7a9a5ca49 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:59.191817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6033c85-11e0-4fdf-8533-d4c7e8b8f302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:59.192444 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6033c85-11e0-4fdf-8533-d4c7e8b8f302 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:59.192723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6033c85-11e0-4fdf-8533-d4c7e8b8f302 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:59.192994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6033c85-11e0-4fdf-8533-d4c7e8b8f302 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:59.193962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 483/982] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:45:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:45:59.195009 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b95928-c867-4de1-9146-66801c42cb59 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:59.206110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6033c85-11e0-4fdf-8533-d4c7e8b8f302 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:59.207640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6033c85-11e0-4fdf-8533-d4c7e8b8f302 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:59.223768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-a6033c85-11e0-4fdf-8533-d4c7e8b8f302 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:59.223768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:59.223768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:59.223768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:59.223768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:59.223768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:59.223768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:59.223768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:59.223768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:59.223768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:59.223768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:59.223768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:59.223768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:45:59.223768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:45:59.223768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:45:59.223768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:45:59.224771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:59.224771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:59.224771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:59.224771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:59.224771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:59.224771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:59.224771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:59.224771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:59.224771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:59.224771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:59.224771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:59.224771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:59.224771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:59.224771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:59.224771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:59.224771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:59.224771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:45:59.226364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:45:59.226364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:45:59.226364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:59.226364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 500/983] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:59.230273 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6033c85-11e0-4fdf-8533-d4c7e8b8f302 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:59.313373 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b95928-c867-4de1-9146-66801c42cb59 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:59.349892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6033c85-11e0-4fdf-8533-d4c7e8b8f302 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:59.354336 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d60af6-cf38-4dd2-8664-260ee4c80715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:59.354986 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d60af6-cf38-4dd2-8664-260ee4c80715 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:59.355273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d60af6-cf38-4dd2-8664-260ee4c80715 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:59.355542 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d60af6-cf38-4dd2-8664-260ee4c80715 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:59.356172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d60af6-cf38-4dd2-8664-260ee4c80715 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:59.356471 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d60af6-cf38-4dd2-8664-260ee4c80715 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:59.358275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d60af6-cf38-4dd2-8664-260ee4c80715 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:59.363914 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 484/984] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Sep 22 14:45:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:59.369494 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d60af6-cf38-4dd2-8664-260ee4c80715 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:59.385369 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:59.387849 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d0fd3-08fa-47b0-840c-dcc4274309ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:59.388523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d0fd3-08fa-47b0-840c-dcc4274309ca None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:59.388799 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d0fd3-08fa-47b0-840c-dcc4274309ca None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:59.389067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d0fd3-08fa-47b0-840c-dcc4274309ca None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:59.390300 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d0fd3-08fa-47b0-840c-dcc4274309ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:59.390658 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d0fd3-08fa-47b0-840c-dcc4274309ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:59.392047 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d0fd3-08fa-47b0-840c-dcc4274309ca None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:59.393659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:59.407575 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 501/985] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:45:59.413048 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d0fd3-08fa-47b0-840c-dcc4274309ca None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:59.478693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d60af6-cf38-4dd2-8664-260ee4c80715 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:59.513015 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46756851-a45c-467d-aef0-d3c7fb29141e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:59.513614 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46756851-a45c-467d-aef0-d3c7fb29141e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:59.513883 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46756851-a45c-467d-aef0-d3c7fb29141e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:59.514147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46756851-a45c-467d-aef0-d3c7fb29141e None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:59.514817 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46756851-a45c-467d-aef0-d3c7fb29141e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:59.515111 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46756851-a45c-467d-aef0-d3c7fb29141e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:59.516460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46756851-a45c-467d-aef0-d3c7fb29141e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:59.518092 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d0fd3-08fa-47b0-840c-dcc4274309ca None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:59.521275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 485/986] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Sep 22 14:45:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:59.522058 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46756851-a45c-467d-aef0-d3c7fb29141e None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:59.554392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658a3e8-9e3b-43f5-b48c-582bff3f7b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:59.555067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658a3e8-9e3b-43f5-b48c-582bff3f7b65 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:59.555349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658a3e8-9e3b-43f5-b48c-582bff3f7b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:59.555626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658a3e8-9e3b-43f5-b48c-582bff3f7b65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:59.562250 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658a3e8-9e3b-43f5-b48c-582bff3f7b65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:59.563711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658a3e8-9e3b-43f5-b48c-582bff3f7b65 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:59.574787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-f658a3e8-9e3b-43f5-b48c-582bff3f7b65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:59.574787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:59.574787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:59.574787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:59.574787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:59.574787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:59.574787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:59.574787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:59.574787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:59.574787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:59.574787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:59.574787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:59.574787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:45:59.574787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:45:59.574787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:45:59.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:45:59.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:59.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:59.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:59.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:59.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:59.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:59.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:59.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:59.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:59.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:59.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:59.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:59.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:59.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:59.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:59.575740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:59.576640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:45:59.576640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:45:59.576640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:45:59.576640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:45:59.576640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:45:59.576640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:45:59.576640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:45:59.576640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:45:59.576640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 502/987] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:45:59.577669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658a3e8-9e3b-43f5-b48c-582bff3f7b65 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:59.601888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:59.624921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46756851-a45c-467d-aef0-d3c7fb29141e None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:59.636535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:59.659749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2a150-837d-479a-b7d0-6ce91f94ea4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:59.660438 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2a150-837d-479a-b7d0-6ce91f94ea4c None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:59.660654 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2a150-837d-479a-b7d0-6ce91f94ea4c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:59.660950 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2a150-837d-479a-b7d0-6ce91f94ea4c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:59.661505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2a150-837d-479a-b7d0-6ce91f94ea4c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:59.662897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2a150-837d-479a-b7d0-6ce91f94ea4c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:59.671839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-c2c2a150-837d-479a-b7d0-6ce91f94ea4c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:45:59.671839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:59.671839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:45:59.671839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:45:59.671839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:45:59.671839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:45:59.671839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:45:59.671839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:59.671839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:45:59.671839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:59.671839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:45:59.671839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:59.671839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:45:59.671839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:45:59.671839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:45:59.671839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:45:59.671839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:45:59.672862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:45:59.672862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:45:59.672862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:45:59.672862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:45:59.672862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:45:59.672862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:45:59.672862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:45:59.672862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:45:59.672862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:45:59.672862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:45:59.672862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:45:59.672862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:45:59.672862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:45:59.672862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:45:59.672862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:45:59.672862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:45:59.673709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:45:59.673709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:45:59.673709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:45:59.673709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:45:59.673709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:45:59.673709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:45:59.673709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:45:59.673709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:45:59.673709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:45:59.673709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:45:59.673709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:45:59.673709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:45:59.673709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:45:59.673709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:45:59.673709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 486/988] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:45:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:45:59.674516 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2a150-837d-479a-b7d0-6ce91f94ea4c None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:59.688760 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658a3e8-9e3b-43f5-b48c-582bff3f7b65 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:59.723707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a89d75-e83f-4e1c-84ef-9f4fec6ac906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:59.724319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a89d75-e83f-4e1c-84ef-9f4fec6ac906 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:59.724600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a89d75-e83f-4e1c-84ef-9f4fec6ac906 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:59.724874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a89d75-e83f-4e1c-84ef-9f4fec6ac906 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:59.725580 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a89d75-e83f-4e1c-84ef-9f4fec6ac906 None admin] RBAC: Adding query filter params (interface=admin, service_id=81012ce1e221458f8d3eb92140e69d50, region_id=RegionOne) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:59.725890 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a89d75-e83f-4e1c-84ef-9f4fec6ac906 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:59.727299 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a89d75-e83f-4e1c-84ef-9f4fec6ac906 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:59.732339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 503/989] 10.222.0.26 () {56 vars in 1249 bytes} [Thu Sep 22 14:45:59 2022] GET /identity/v3/endpoints?service_id=81012ce1e221458f8d3eb92140e69d50&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:59.733131 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a89d75-e83f-4e1c-84ef-9f4fec6ac906 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:59.773482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2a150-837d-479a-b7d0-6ce91f94ea4c None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:59.806865 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94f85e-28cd-4ef9-b985-be2b3f4ad04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:59.807465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94f85e-28cd-4ef9-b985-be2b3f4ad04f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:59.807746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94f85e-28cd-4ef9-b985-be2b3f4ad04f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:59.808015 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94f85e-28cd-4ef9-b985-be2b3f4ad04f None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:59.808693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94f85e-28cd-4ef9-b985-be2b3f4ad04f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:59.808991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94f85e-28cd-4ef9-b985-be2b3f4ad04f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:59.810419 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94f85e-28cd-4ef9-b985-be2b3f4ad04f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:59.822951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 487/990] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:45:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:59.823774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94f85e-28cd-4ef9-b985-be2b3f4ad04f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:59.826597 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:59.833317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a89d75-e83f-4e1c-84ef-9f4fec6ac906 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:59.851772 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:45:59.868451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ec9ca-7def-48f4-8d5f-1c4eafa9b627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:59.868980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ec9ca-7def-48f4-8d5f-1c4eafa9b627 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:59.869250 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ec9ca-7def-48f4-8d5f-1c4eafa9b627 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:59.869514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ec9ca-7def-48f4-8d5f-1c4eafa9b627 None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:59.870128 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ec9ca-7def-48f4-8d5f-1c4eafa9b627 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:45:59.870426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ec9ca-7def-48f4-8d5f-1c4eafa9b627 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:59.871809 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ec9ca-7def-48f4-8d5f-1c4eafa9b627 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:59.876595 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 504/991] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:45:59.877584 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ec9ca-7def-48f4-8d5f-1c4eafa9b627 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:59.928528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94f85e-28cd-4ef9-b985-be2b3f4ad04f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:45:59.964853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5ff5e-0e7e-4e9d-b24b-58875cf868db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:45:59.965451 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5ff5e-0e7e-4e9d-b24b-58875cf868db None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:45:59.965725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5ff5e-0e7e-4e9d-b24b-58875cf868db None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:45:59.965991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5ff5e-0e7e-4e9d-b24b-58875cf868db None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:45:59.966591 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5ff5e-0e7e-4e9d-b24b-58875cf868db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:45:59.967999 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5ff5e-0e7e-4e9d-b24b-58875cf868db None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:45:59.974389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 488/992] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Sep 22 14:45:59 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:45:59.975195 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5ff5e-0e7e-4e9d-b24b-58875cf868db None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:45:59.987542 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ec9ca-7def-48f4-8d5f-1c4eafa9b627 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:00.029361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccba949-f34a-4f6a-9ec9-9f677e04e975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:00.030210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccba949-f34a-4f6a-9ec9-9f677e04e975 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:00.030527 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccba949-f34a-4f6a-9ec9-9f677e04e975 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:00.031199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccba949-f34a-4f6a-9ec9-9f677e04e975 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:00.031934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccba949-f34a-4f6a-9ec9-9f677e04e975 None admin] RBAC: Adding query filter params (role.id=4314083fe9cb43b3a969c8567ddef4c7, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=6256407d9df648d5bbe33495cb9a8b36) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:00.032267 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccba949-f34a-4f6a-9ec9-9f677e04e975 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:00.033719 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccba949-f34a-4f6a-9ec9-9f677e04e975 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:00.041531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 505/993] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:45:59 2022] GET /identity/v3/role_assignments?user.id=6256407d9df648d5bbe33495cb9a8b36&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=4314083fe9cb43b3a969c8567ddef4c7 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:00.056662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:00.103707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5ff5e-0e7e-4e9d-b24b-58875cf868db None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:00.147129 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c60e54-8f06-42de-b0de-807bf8eb41a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:00.147674 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c60e54-8f06-42de-b0de-807bf8eb41a3 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:00.147953 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c60e54-8f06-42de-b0de-807bf8eb41a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:00.148400 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c60e54-8f06-42de-b0de-807bf8eb41a3 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:00.149150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c60e54-8f06-42de-b0de-807bf8eb41a3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:00.149462 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c60e54-8f06-42de-b0de-807bf8eb41a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:00.150917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c60e54-8f06-42de-b0de-807bf8eb41a3 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:00.161963 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 489/994] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Sep 22 14:45:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:00.163768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccba949-f34a-4f6a-9ec9-9f677e04e975 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:00.218530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:00.279736 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccba949-f34a-4f6a-9ec9-9f677e04e975 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:00.320604 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f52ad-77b5-41e6-b10c-d8f74b21a7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:00.321220 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f52ad-77b5-41e6-b10c-d8f74b21a7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:00.321496 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f52ad-77b5-41e6-b10c-d8f74b21a7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:00.321767 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f52ad-77b5-41e6-b10c-d8f74b21a7b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:00.323097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f52ad-77b5-41e6-b10c-d8f74b21a7b5 None admin] RBAC: Adding query filter params (interface=public, service_id=82483fcb91324806a605fd785ce5d869, region_id=RegionOne) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:00.323097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f52ad-77b5-41e6-b10c-d8f74b21a7b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:00.325425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f52ad-77b5-41e6-b10c-d8f74b21a7b5 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:00.334821 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 506/995] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Sep 22 14:46:00 2022] GET /identity/v3/endpoints?service_id=82483fcb91324806a605fd785ce5d869&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:00.336123 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c60e54-8f06-42de-b0de-807bf8eb41a3 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:00.440082 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c60e54-8f06-42de-b0de-807bf8eb41a3 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:00.469897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:00.477100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794fab2-3416-431d-b4d2-76508b4f6a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:00.477708 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794fab2-3416-431d-b4d2-76508b4f6a58 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:00.477979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794fab2-3416-431d-b4d2-76508b4f6a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:00.478246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794fab2-3416-431d-b4d2-76508b4f6a58 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:00.478875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794fab2-3416-431d-b4d2-76508b4f6a58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:00.480232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794fab2-3416-431d-b4d2-76508b4f6a58 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:00.487228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 490/996] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Sep 22 14:46:00 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:46:02.220934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266839ff-4e21-4f50-bb20-33a3e9bdef1b None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:02.221508 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266839ff-4e21-4f50-bb20-33a3e9bdef1b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:02.221508 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266839ff-4e21-4f50-bb20-33a3e9bdef1b None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:02.223153 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 507/997] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:02.228390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977b180-ba5f-4901-a9f8-76f7fcb6f188 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:02.228792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977b180-ba5f-4901-a9f8-76f7fcb6f188 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:02.229001 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977b180-ba5f-4901-a9f8-76f7fcb6f188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:02.247245 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-8977b180-ba5f-4901-a9f8-76f7fcb6f188 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:02.248438 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-8977b180-ba5f-4901-a9f8-76f7fcb6f188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:02.269971 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8002fba-db37-4bd2-9676-56c6ab1a77ca None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:02.270324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8002fba-db37-4bd2-9676-56c6ab1a77ca None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:02.270644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8002fba-db37-4bd2-9676-56c6ab1a77ca None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:02.272050 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 508/998] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:46:02.277119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badcd11-b4ea-4f15-8d6b-378b33363d4a None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:02.277406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badcd11-b4ea-4f15-8d6b-378b33363d4a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:02.277688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badcd11-b4ea-4f15-8d6b-378b33363d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:02.296199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-2badcd11-b4ea-4f15-8d6b-378b33363d4a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:02.297373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-2badcd11-b4ea-4f15-8d6b-378b33363d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:02.330875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 491/999] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:02.372488 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 509/1000] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:46:02.606598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977b180-ba5f-4901-a9f8-76f7fcb6f188 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:02.641091 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badcd11-b4ea-4f15-8d6b-378b33363d4a None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:02.686907 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:02.712575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977b180-ba5f-4901-a9f8-76f7fcb6f188 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:02.736447 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:02.745698 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badcd11-b4ea-4f15-8d6b-378b33363d4a None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:02.747382 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0d162-08dc-4c0a-88dc-5be3b5e35cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:02.748013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0d162-08dc-4c0a-88dc-5be3b5e35cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:02.748352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0d162-08dc-4c0a-88dc-5be3b5e35cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:02.748623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0d162-08dc-4c0a-88dc-5be3b5e35cb5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:02.749231 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0d162-08dc-4c0a-88dc-5be3b5e35cb5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:02.750589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0d162-08dc-4c0a-88dc-5be3b5e35cb5 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:02.759313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-47e0d162-08dc-4c0a-88dc-5be3b5e35cb5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:46:02.759313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:02.759313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:46:02.759313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:46:02.759313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:46:02.759313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:46:02.759313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:46:02.759313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:02.759313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:46:02.759313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:02.759313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:46:02.759313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:02.759313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:02.759313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:02.759313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:02.759313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:02.759313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:02.760510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:02.760510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:02.760510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:02.760510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:02.760510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:02.760510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:02.760510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:46:02.760510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:46:02.760510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:46:02.760510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:46:02.760510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:02.760510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:02.760510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:02.760510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:02.760510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:02.760510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:02.761631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:02.761631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:02.761631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:02.761631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:02.761631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:02.761631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:02.761631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:02.761631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:02.761631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:02.761631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:02.761631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:46:02.761631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:02.761631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 14:46:02.761631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:02.761631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 492/1001] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Sep 22 14:46:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:02.762499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d84e30-1d5c-468e-9ca4-3063600223f5 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:02.762499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d84e30-1d5c-468e-9ca4-3063600223f5 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:02.762499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d84e30-1d5c-468e-9ca4-3063600223f5 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:02.763292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 493/1002] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:02.766663 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d84e30-1d5c-468e-9ca4-3063600223f5 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:02.786156 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152b8b1-71fd-44eb-8f11-c04a081a8347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:02.786817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152b8b1-71fd-44eb-8f11-c04a081a8347 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:02.787100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152b8b1-71fd-44eb-8f11-c04a081a8347 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:02.787372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152b8b1-71fd-44eb-8f11-c04a081a8347 None admin] PATH_INFO: `/v3/services/image` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:02.787950 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152b8b1-71fd-44eb-8f11-c04a081a8347 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:02.789308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152b8b1-71fd-44eb-8f11-c04a081a8347 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:02.798255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-c152b8b1-71fd-44eb-8f11-c04a081a8347 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:02.798255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:02.798255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:46:02.798255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:46:02.798255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:46:02.798255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:46:02.798255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:46:02.798255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:02.798255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:02.798255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:02.798255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:46:02.798255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:02.798255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:02.798255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:02.798255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:02.798255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:02.798255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:02.799570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:02.799570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:02.799570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:02.799570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:02.799570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:02.799570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:02.799570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:46:02.799570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:46:02.799570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:46:02.799570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:46:02.799570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:02.799570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:02.799570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:02.799570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:02.799570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:02.799570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:02.800566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:02.800566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:02.800566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:02.800566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:02.800566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:02.800566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:02.800566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:02.800566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:02.800566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:02.800566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:02.800566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:46:02.800566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:02.800566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:02.800566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:02.800566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 510/1003] 10.222.0.26 () {56 vars in 1105 bytes} [Thu Sep 22 14:46:02 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:02.801372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328b9cd-abca-435d-b067-7d982e5a5a91 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:02.801372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328b9cd-abca-435d-b067-7d982e5a5a91 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:02.801372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328b9cd-abca-435d-b067-7d982e5a5a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:02.818944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-b328b9cd-abca-435d-b067-7d982e5a5a91 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:02.820143 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b328b9cd-abca-435d-b067-7d982e5a5a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:02.873353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d84e30-1d5c-468e-9ca4-3063600223f5 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:02.890827 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 511/1004] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:02.891683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328b9cd-abca-435d-b067-7d982e5a5a91 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:02.908504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a8363-279e-4c81-aa06-b403cb7e89d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:02.909114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a8363-279e-4c81-aa06-b403cb7e89d1 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:02.909402 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a8363-279e-4c81-aa06-b403cb7e89d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:02.909671 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a8363-279e-4c81-aa06-b403cb7e89d1 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:02.910295 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a8363-279e-4c81-aa06-b403cb7e89d1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:02.910599 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a8363-279e-4c81-aa06-b403cb7e89d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:02.912004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a8363-279e-4c81-aa06-b403cb7e89d1 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:02.921320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 494/1005] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Sep 22 14:46:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:02.922316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a8363-279e-4c81-aa06-b403cb7e89d1 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:02.927691 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:02.983803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:03.005379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328b9cd-abca-435d-b067-7d982e5a5a91 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:03.033365 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a8363-279e-4c81-aa06-b403cb7e89d1 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:03.044450 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a9c3e-aac2-41de-b8b9-e515d424e26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:03.045183 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a9c3e-aac2-41de-b8b9-e515d424e26d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:03.045555 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a9c3e-aac2-41de-b8b9-e515d424e26d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:03.045839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a9c3e-aac2-41de-b8b9-e515d424e26d None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:03.046582 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a9c3e-aac2-41de-b8b9-e515d424e26d None admin] RBAC: Adding query filter params (name=image) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:03.047032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a9c3e-aac2-41de-b8b9-e515d424e26d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:03.048393 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a9c3e-aac2-41de-b8b9-e515d424e26d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:03.058187 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 512/1006] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:03.059484 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a9c3e-aac2-41de-b8b9-e515d424e26d None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:03.072449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e408133-5c1d-49ee-bae7-910a958d61ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:03.073158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e408133-5c1d-49ee-bae7-910a958d61ab None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:03.073442 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e408133-5c1d-49ee-bae7-910a958d61ab None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:03.073724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e408133-5c1d-49ee-bae7-910a958d61ab None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:03.074351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e408133-5c1d-49ee-bae7-910a958d61ab None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:03.074655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e408133-5c1d-49ee-bae7-910a958d61ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:03.076061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e408133-5c1d-49ee-bae7-910a958d61ab None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:03.081829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 495/1007] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Sep 22 14:46:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:03.087372 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e408133-5c1d-49ee-bae7-910a958d61ab None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:03.169885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:03.174204 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a9c3e-aac2-41de-b8b9-e515d424e26d None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:03.188738 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e408133-5c1d-49ee-bae7-910a958d61ab None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:03.205258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:03.211328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2aa7d8-2363-4019-b25f-b4f37b4ec76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:03.211930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2aa7d8-2363-4019-b25f-b4f37b4ec76d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:03.212205 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2aa7d8-2363-4019-b25f-b4f37b4ec76d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:03.212473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2aa7d8-2363-4019-b25f-b4f37b4ec76d None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:03.213099 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2aa7d8-2363-4019-b25f-b4f37b4ec76d None admin] RBAC: Adding query filter params (type=image) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:03.213400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2aa7d8-2363-4019-b25f-b4f37b4ec76d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:03.214771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2aa7d8-2363-4019-b25f-b4f37b4ec76d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:03.219611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 513/1008] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:03.220388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2aa7d8-2363-4019-b25f-b4f37b4ec76d None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:03.237934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22553e-6a5b-4bf4-a9ef-b34afbecaef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:03.238785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22553e-6a5b-4bf4-a9ef-b34afbecaef2 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:03.239168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22553e-6a5b-4bf4-a9ef-b34afbecaef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:03.239519 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22553e-6a5b-4bf4-a9ef-b34afbecaef2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:03.240601 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22553e-6a5b-4bf4-a9ef-b34afbecaef2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:03.241752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22553e-6a5b-4bf4-a9ef-b34afbecaef2 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:03.261181 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 496/1009] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:46:03 2022] POST /identity/v3/endpoints => generated 325 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:46:03.351629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2aa7d8-2363-4019-b25f-b4f37b4ec76d None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:03.406305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8cc4fa-532a-4d5b-8054-dbd63a858bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:03.407008 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cc4fa-532a-4d5b-8054-dbd63a858bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:03.407299 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cc4fa-532a-4d5b-8054-dbd63a858bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:03.407546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cc4fa-532a-4d5b-8054-dbd63a858bc2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:03.408133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8cc4fa-532a-4d5b-8054-dbd63a858bc2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:03.409567 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8cc4fa-532a-4d5b-8054-dbd63a858bc2 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:03.420528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-3f8cc4fa-532a-4d5b-8054-dbd63a858bc2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:46:03.420528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:03.420528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:46:03.420528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:46:03.420528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:46:03.420528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:46:03.420528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:46:03.420528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:03.420528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:46:03.420528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:03.420528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:46:03.420528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:03.420528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:03.420528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:03.420528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:03.420528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:03.420528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:03.421645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:03.421645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:03.421645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:03.421645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:03.421645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:03.421645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:03.421645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:46:03.421645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:46:03.421645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:46:03.421645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:46:03.421645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:03.421645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:03.421645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:03.421645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:03.421645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:03.421645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:03.422536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:03.422536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:03.422536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:03.422536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:03.422536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:03.422536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:03.422536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:03.422536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:03.422536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:03.422536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:03.422536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:46:03.422536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:03.422536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 14:46:03.422536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:03.422536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 514/1010] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:46:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:03.428260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22553e-6a5b-4bf4-a9ef-b34afbecaef2 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:03.541390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22553e-6a5b-4bf4-a9ef-b34afbecaef2 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:03.580956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4ced7-1b0d-4393-91b9-58472720b19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:03.581578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4ced7-1b0d-4393-91b9-58472720b19f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:03.581857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4ced7-1b0d-4393-91b9-58472720b19f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:03.582126 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4ced7-1b0d-4393-91b9-58472720b19f None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:03.582785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4ced7-1b0d-4393-91b9-58472720b19f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:03.583092 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4ced7-1b0d-4393-91b9-58472720b19f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:03.584458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4ced7-1b0d-4393-91b9-58472720b19f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:03.590028 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 497/1011] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Sep 22 14:46:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:03.595720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8cc4fa-532a-4d5b-8054-dbd63a858bc2 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:03.622440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:03.701953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8cc4fa-532a-4d5b-8054-dbd63a858bc2 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:03.739447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda22fa-5574-4e9c-b880-569f11e7f7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:03.740051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda22fa-5574-4e9c-b880-569f11e7f7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:03.740323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda22fa-5574-4e9c-b880-569f11e7f7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:03.740590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda22fa-5574-4e9c-b880-569f11e7f7b8 None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:03.741210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda22fa-5574-4e9c-b880-569f11e7f7b8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:03.741506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda22fa-5574-4e9c-b880-569f11e7f7b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:03.742889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda22fa-5574-4e9c-b880-569f11e7f7b8 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:03.752624 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 515/1012] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Sep 22 14:46:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:03.753789 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4ced7-1b0d-4393-91b9-58472720b19f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:03.797723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:03.859658 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4ced7-1b0d-4393-91b9-58472720b19f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:03.898789 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf052b81-0da7-4c45-82f4-3753b2db9d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:03.899447 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf052b81-0da7-4c45-82f4-3753b2db9d22 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:03.899680 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf052b81-0da7-4c45-82f4-3753b2db9d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:03.899947 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf052b81-0da7-4c45-82f4-3753b2db9d22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:03.900694 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf052b81-0da7-4c45-82f4-3753b2db9d22 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:03.901979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf052b81-0da7-4c45-82f4-3753b2db9d22 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:03.934769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 498/1013] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:46:03 2022] POST /identity/v3/endpoints => generated 334 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:46:05.431323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f3c0c-59ac-4bba-a410-8002258c21dd None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:05.431888 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f3c0c-59ac-4bba-a410-8002258c21dd None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:05.431888 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f3c0c-59ac-4bba-a410-8002258c21dd None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:05.433267 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 516/1014] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:05.438846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb7935-25d2-49ec-97ff-ad2f7ee903cc None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:05.439270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb7935-25d2-49ec-97ff-ad2f7ee903cc None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:05.439455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb7935-25d2-49ec-97ff-ad2f7ee903cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:05.457648 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-8eeb7935-25d2-49ec-97ff-ad2f7ee903cc None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:05.458866 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-8eeb7935-25d2-49ec-97ff-ad2f7ee903cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:05.524186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe8721-70ed-4dda-8422-7d286a9b8246 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:05.524467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe8721-70ed-4dda-8422-7d286a9b8246 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:05.524740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe8721-70ed-4dda-8422-7d286a9b8246 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:05.526080 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 517/1015] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:05.532323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9a257-73ca-4c1f-877e-97775bc4ae5e None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:05.532723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9a257-73ca-4c1f-877e-97775bc4ae5e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:05.533093 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9a257-73ca-4c1f-877e-97775bc4ae5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:05.534109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 499/1016] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:05 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:05.552480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-52f9a257-73ca-4c1f-877e-97775bc4ae5e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:05.553672 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-52f9a257-73ca-4c1f-877e-97775bc4ae5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:05.623844 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 518/1017] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:05 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:46:05.806355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb7935-25d2-49ec-97ff-ad2f7ee903cc None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:05.830543 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:05.892794 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9a257-73ca-4c1f-877e-97775bc4ae5e None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:05.911378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:05.911766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb7935-25d2-49ec-97ff-ad2f7ee903cc None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:05.947334 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4fb19-159a-47fc-85fe-54399b0aa9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:05.947945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4fb19-159a-47fc-85fe-54399b0aa9f8 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:05.948225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4fb19-159a-47fc-85fe-54399b0aa9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:05.948496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4fb19-159a-47fc-85fe-54399b0aa9f8 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:05.949158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4fb19-159a-47fc-85fe-54399b0aa9f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:05.950539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4fb19-159a-47fc-85fe-54399b0aa9f8 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:05.973396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 500/1018] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Sep 22 14:46:05 2022] POST /identity/v3/services => generated 242 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:46:06.017271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9a257-73ca-4c1f-877e-97775bc4ae5e None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:06.065405 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869462c-7c0d-416a-bb8c-4912d74e26ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:06.066546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869462c-7c0d-416a-bb8c-4912d74e26ca None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:06.067101 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869462c-7c0d-416a-bb8c-4912d74e26ca None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:06.067592 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869462c-7c0d-416a-bb8c-4912d74e26ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:06.069233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869462c-7c0d-416a-bb8c-4912d74e26ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:06.071244 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869462c-7c0d-416a-bb8c-4912d74e26ca None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:06.073350 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 519/1019] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:06.080485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4fb19-159a-47fc-85fe-54399b0aa9f8 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:06.176196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:06.179036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ec504-d178-4d84-9e8f-38504e0bcb68 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:06.179324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ec504-d178-4d84-9e8f-38504e0bcb68 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:06.179597 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ec504-d178-4d84-9e8f-38504e0bcb68 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:06.181091 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 520/1020] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:46:06.186860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74550d7b-c2e2-4d74-985a-976a21290224 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:06.187147 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74550d7b-c2e2-4d74-985a-976a21290224 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:06.187428 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74550d7b-c2e2-4d74-985a-976a21290224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:06.199116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4fb19-159a-47fc-85fe-54399b0aa9f8 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:06.206963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-74550d7b-c2e2-4d74-985a-976a21290224 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:06.208192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-74550d7b-c2e2-4d74-985a-976a21290224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:06.236746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55bbaf-bc05-46eb-847b-6a381dc3879f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:06.237712 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55bbaf-bc05-46eb-847b-6a381dc3879f None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:06.238204 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55bbaf-bc05-46eb-847b-6a381dc3879f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:06.238698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55bbaf-bc05-46eb-847b-6a381dc3879f None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:06.239654 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55bbaf-bc05-46eb-847b-6a381dc3879f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:06.241261 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55bbaf-bc05-46eb-847b-6a381dc3879f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:06.271536 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 501/1021] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Sep 22 14:46:06 2022] POST /identity/v3/projects => generated 311 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:46:06.279275 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 521/1022] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:46:06.556211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55bbaf-bc05-46eb-847b-6a381dc3879f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:06.662713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55bbaf-bc05-46eb-847b-6a381dc3879f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:06.698041 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e70dff-7317-49ea-9201-fa606a53c963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:06.698680 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e70dff-7317-49ea-9201-fa606a53c963 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:06.699000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e70dff-7317-49ea-9201-fa606a53c963 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:06.699276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e70dff-7317-49ea-9201-fa606a53c963 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:06.701125 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:06.713184 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e70dff-7317-49ea-9201-fa606a53c963 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:06.718076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-16e70dff-7317-49ea-9201-fa606a53c963 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:46:06.718076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:06.718076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:06.718076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:06.718076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:06.718076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:06.718076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:06.718076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:06.718076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:06.718076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:06.718076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:06.718076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:06.718076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:06.718076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:06.718076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:06.718076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:06.719255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:06.719255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:06.719255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:06.719255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:06.719255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:06.719255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:06.719255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:06.719255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:06.719255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:06.719255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:06.719255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:06.719255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:06.719255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:06.719255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:06.719255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:06.719255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:06.720132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:06.720132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:06.720132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:06.720132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:06.720132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:06.720132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:06.720132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:06.720132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:06.720132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:06.720132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:06.720132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:46:06.720132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:06.721292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 502/1023] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Sep 22 14:46:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:06.725662 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74550d7b-c2e2-4d74-985a-976a21290224 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:06.737756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:06.832004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74550d7b-c2e2-4d74-985a-976a21290224 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:06.868255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b615048-4b92-4920-b5b8-f6e19d48aaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:06.868928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b615048-4b92-4920-b5b8-f6e19d48aaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:06.869212 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b615048-4b92-4920-b5b8-f6e19d48aaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:06.869487 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b615048-4b92-4920-b5b8-f6e19d48aaa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:06.870149 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b615048-4b92-4920-b5b8-f6e19d48aaa6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:06.870454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b615048-4b92-4920-b5b8-f6e19d48aaa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:06.871845 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b615048-4b92-4920-b5b8-f6e19d48aaa6 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:06.885708 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 522/1024] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Sep 22 14:46:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:06.887155 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e70dff-7317-49ea-9201-fa606a53c963 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:06.992767 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e70dff-7317-49ea-9201-fa606a53c963 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:07.027521 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96fc44-212b-4038-9201-fb8285ea7c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:07.028158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96fc44-212b-4038-9201-fb8285ea7c70 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:07.028436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96fc44-212b-4038-9201-fb8285ea7c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:07.028706 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96fc44-212b-4038-9201-fb8285ea7c70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:07.040017 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef96fc44-212b-4038-9201-fb8285ea7c70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:07.041539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef96fc44-212b-4038-9201-fb8285ea7c70 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:07.056902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-ef96fc44-212b-4038-9201-fb8285ea7c70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:07.056902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:07.056902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:07.056902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:07.056902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:07.056902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:07.056902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:07.056902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:07.056902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:07.056902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:07.056902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:07.056902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:07.056902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:46:07.056902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:46:07.056902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:46:07.056902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:46:07.057910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:07.057910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:07.057910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:07.057910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:07.057910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:07.057910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:07.057910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:07.057910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:07.057910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:07.057910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:07.057910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:07.057910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:07.057910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:07.057910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:07.057910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:07.057910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:07.057910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:46:07.058864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:46:07.058864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:07.058864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:07.058864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 503/1025] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:07.065253 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b615048-4b92-4920-b5b8-f6e19d48aaa6 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:07.143504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:07.175766 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b615048-4b92-4920-b5b8-f6e19d48aaa6 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:07.211197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ce724-c8af-4197-9ba0-1b5c78ab827e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:07.211636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ce724-c8af-4197-9ba0-1b5c78ab827e None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:07.211912 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ce724-c8af-4197-9ba0-1b5c78ab827e None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:07.212180 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ce724-c8af-4197-9ba0-1b5c78ab827e None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:07.212821 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ce724-c8af-4197-9ba0-1b5c78ab827e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:07.213122 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ce724-c8af-4197-9ba0-1b5c78ab827e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:07.214482 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ce724-c8af-4197-9ba0-1b5c78ab827e None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:07.230782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 523/1026] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:07.233880 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96fc44-212b-4038-9201-fb8285ea7c70 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:07.285150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:07.347892 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96fc44-212b-4038-9201-fb8285ea7c70 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:07.381434 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3da02a-6f83-455d-ba31-a27672bc1085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:07.382057 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3da02a-6f83-455d-ba31-a27672bc1085 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:07.382329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3da02a-6f83-455d-ba31-a27672bc1085 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:07.382592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3da02a-6f83-455d-ba31-a27672bc1085 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:07.390495 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3da02a-6f83-455d-ba31-a27672bc1085 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:07.391950 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3da02a-6f83-455d-ba31-a27672bc1085 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:07.403114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-3e3da02a-6f83-455d-ba31-a27672bc1085 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:46:07.403114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:07.403114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:07.403114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:07.403114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:07.403114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:07.403114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:07.403114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:07.403114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:07.403114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:07.403114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:07.403114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:07.403114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:46:07.403114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:46:07.403114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:46:07.403114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:07.404097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:07.410404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:07.410404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:07.410404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:46:07.410404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:46:07.410404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:46:07.410404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:46:07.410404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:46:07.410404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:46:07.410404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:46:07.410404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:07.410404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 504/1027] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Sep 22 14:46:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:07.411068 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ce724-c8af-4197-9ba0-1b5c78ab827e None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:07.511236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ce724-c8af-4197-9ba0-1b5c78ab827e None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:07.540757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:07.547125 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e47ad-3f8d-48c6-8568-1259cebf8d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:07.547733 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e47ad-3f8d-48c6-8568-1259cebf8d01 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:07.548006 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e47ad-3f8d-48c6-8568-1259cebf8d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:07.548277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e47ad-3f8d-48c6-8568-1259cebf8d01 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:07.548994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e47ad-3f8d-48c6-8568-1259cebf8d01 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:07.549289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e47ad-3f8d-48c6-8568-1259cebf8d01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:07.550701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e47ad-3f8d-48c6-8568-1259cebf8d01 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:07.573412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 524/1028] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Sep 22 14:46:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:07.574611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3da02a-6f83-455d-ba31-a27672bc1085 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:07.681542 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3da02a-6f83-455d-ba31-a27672bc1085 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:07.689667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:07.718990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cf1ad-a8ae-4ef5-8bcb-4bc64d878ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:07.719604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cf1ad-a8ae-4ef5-8bcb-4bc64d878ecf None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:07.719883 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cf1ad-a8ae-4ef5-8bcb-4bc64d878ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:07.720154 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cf1ad-a8ae-4ef5-8bcb-4bc64d878ecf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:07.789923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675cf1ad-a8ae-4ef5-8bcb-4bc64d878ecf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:07.791417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675cf1ad-a8ae-4ef5-8bcb-4bc64d878ecf None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:07.900984 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 505/1029] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:07 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:46:07.906994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e47ad-3f8d-48c6-8568-1259cebf8d01 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:08.011598 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e47ad-3f8d-48c6-8568-1259cebf8d01 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:08.047495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36936cec-1773-4a6c-9404-e7896405a13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:08.048132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936cec-1773-4a6c-9404-e7896405a13a None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.048413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936cec-1773-4a6c-9404-e7896405a13a None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.048686 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36936cec-1773-4a6c-9404-e7896405a13a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.059502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36936cec-1773-4a6c-9404-e7896405a13a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:08.060950 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36936cec-1773-4a6c-9404-e7896405a13a None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:08.077218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-36936cec-1773-4a6c-9404-e7896405a13a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:08.077218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:08.077218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:08.077218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:08.077218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:08.077218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:08.077218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:08.077218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:08.077218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:08.077218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:08.077218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:08.077218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:08.077218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:46:08.077218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:46:08.077218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:46:08.077218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:46:08.078252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:08.078252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:08.078252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:08.078252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:08.078252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:08.078252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:08.078252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:08.078252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:08.078252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:08.078252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:08.078252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:08.078252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:08.078252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:08.078252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:08.078252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:08.078252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:08.078252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:46:08.083624 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:46:08.083624 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:08.083624 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:08.083624 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 525/1030] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:08.086008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cf1ad-a8ae-4ef5-8bcb-4bc64d878ecf None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:08.096337 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:08.194255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cf1ad-a8ae-4ef5-8bcb-4bc64d878ecf None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:08.203527 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b49ea-837a-4e9d-9449-2a990adddef3 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.203811 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b49ea-837a-4e9d-9449-2a990adddef3 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.204076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b49ea-837a-4e9d-9449-2a990adddef3 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.205549 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 526/1031] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:46:08.210580 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb519378-75ed-4394-999c-4838f843ae8d None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.210893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb519378-75ed-4394-999c-4838f843ae8d None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.211164 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb519378-75ed-4394-999c-4838f843ae8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.229555 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-fb519378-75ed-4394-999c-4838f843ae8d None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:08.230529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6029dd2-2d60-445d-9746-0918722848e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:08.230771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-fb519378-75ed-4394-999c-4838f843ae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:08.231169 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6029dd2-2d60-445d-9746-0918722848e3 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.231447 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6029dd2-2d60-445d-9746-0918722848e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.231714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6029dd2-2d60-445d-9746-0918722848e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.232341 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6029dd2-2d60-445d-9746-0918722848e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:08.232813 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6029dd2-2d60-445d-9746-0918722848e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:08.235062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6029dd2-2d60-445d-9746-0918722848e3 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:08.248464 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 506/1032] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:08.254498 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6029dd2-2d60-445d-9746-0918722848e3 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:08.274439 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:08.309026 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 527/1033] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:46:08.332046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:08.368717 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6029dd2-2d60-445d-9746-0918722848e3 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:08.409066 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517ace5-4766-4908-a912-df1428d8c03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:08.409699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517ace5-4766-4908-a912-df1428d8c03e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.409976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517ace5-4766-4908-a912-df1428d8c03e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.410247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517ace5-4766-4908-a912-df1428d8c03e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.417508 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517ace5-4766-4908-a912-df1428d8c03e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:08.418989 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517ace5-4766-4908-a912-df1428d8c03e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:08.430897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-4517ace5-4766-4908-a912-df1428d8c03e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:46:08.430897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:08.430897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:08.430897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:08.430897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:08.430897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:08.430897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:08.430897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:08.430897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:08.430897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:08.430897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:08.430897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:08.430897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:46:08.430897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:46:08.430897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:46:08.431777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:46:08.431777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:08.431777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:08.431777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:08.431777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:08.431777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:08.431777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:08.431777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:08.431777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:08.431777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:08.431777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:08.431777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:08.431777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:08.431777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:08.431777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:08.431777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:08.431777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:08.432726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:46:08.432726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:46:08.432726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:46:08.432726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:46:08.432726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:46:08.432726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:46:08.432726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:46:08.432726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:08.436965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 507/1034] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:46:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:08.437784 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb519378-75ed-4394-999c-4838f843ae8d None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:08.498973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1319c-27ff-44ba-837f-44476c2f1fcf None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.499295 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1319c-27ff-44ba-837f-44476c2f1fcf None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.499517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1319c-27ff-44ba-837f-44476c2f1fcf None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.500861 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 508/1035] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:46:08.506555 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38389efc-a943-4c44-9129-65ac162be5df None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.507252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38389efc-a943-4c44-9129-65ac162be5df None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.507724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38389efc-a943-4c44-9129-65ac162be5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.532228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-38389efc-a943-4c44-9129-65ac162be5df None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:08.534239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-38389efc-a943-4c44-9129-65ac162be5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:08.544354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb519378-75ed-4394-999c-4838f843ae8d None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:08.580782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c85dc0-6c50-424d-b3ed-79c1bef47821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:08.581404 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c85dc0-6c50-424d-b3ed-79c1bef47821 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.581686 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c85dc0-6c50-424d-b3ed-79c1bef47821 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.581956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c85dc0-6c50-424d-b3ed-79c1bef47821 None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.582641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c85dc0-6c50-424d-b3ed-79c1bef47821 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:08.582977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c85dc0-6c50-424d-b3ed-79c1bef47821 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:08.584411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c85dc0-6c50-424d-b3ed-79c1bef47821 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:08.601768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 528/1036] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:46:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:08.601768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c85dc0-6c50-424d-b3ed-79c1bef47821 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:08.616992 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 509/1037] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:08.618469 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38389efc-a943-4c44-9129-65ac162be5df None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:08.627990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:08.688510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:08.704748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c85dc0-6c50-424d-b3ed-79c1bef47821 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:08.741078 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38389efc-a943-4c44-9129-65ac162be5df None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:08.745986 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b843d31-248c-4e29-bf19-294ff005ab65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:08.746621 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b843d31-248c-4e29-bf19-294ff005ab65 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.746930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b843d31-248c-4e29-bf19-294ff005ab65 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.747205 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b843d31-248c-4e29-bf19-294ff005ab65 None admin] PATH_INFO: `/v3/services/image` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.747787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b843d31-248c-4e29-bf19-294ff005ab65 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:08.749304 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b843d31-248c-4e29-bf19-294ff005ab65 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:08.758534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-9b843d31-248c-4e29-bf19-294ff005ab65 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:08.758534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:08.758534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:46:08.758534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:46:08.758534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:46:08.758534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:46:08.758534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:46:08.758534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:08.758534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:08.758534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:08.758534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:46:08.758534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:08.758534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:08.758534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:08.758534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:08.758534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:08.758534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:08.759591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:08.759591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:08.759591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:08.759591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:08.759591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:08.759591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:08.759591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:46:08.759591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:46:08.759591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:46:08.759591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:46:08.759591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:08.759591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:08.759591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:08.759591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:08.759591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:08.759591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:08.760466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:08.760466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:08.760466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:08.760466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:08.760466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:08.760466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:08.760466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:08.760466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:08.760466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:08.760466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:08.760466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:46:08.760466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:08.760466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:08.760466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:08.760466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 529/1038] 10.222.0.26 () {56 vars in 1105 bytes} [Thu Sep 22 14:46:08 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:08.766146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b843d31-248c-4e29-bf19-294ff005ab65 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:08.783380 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d15b5b-1b75-477b-99b5-684d0bd704ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:08.784301 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d15b5b-1b75-477b-99b5-684d0bd704ca None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.784583 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d15b5b-1b75-477b-99b5-684d0bd704ca None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.785000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d15b5b-1b75-477b-99b5-684d0bd704ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.785729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d15b5b-1b75-477b-99b5-684d0bd704ca None admin] RBAC: Adding query filter params (role.id=4314083fe9cb43b3a969c8567ddef4c7, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=0c9be41c39494e54a14398a1f03c4d69) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:08.786070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d15b5b-1b75-477b-99b5-684d0bd704ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:08.787481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d15b5b-1b75-477b-99b5-684d0bd704ca None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:08.792903 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 510/1039] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:46:08 2022] GET /identity/v3/role_assignments?user.id=0c9be41c39494e54a14398a1f03c4d69&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=4314083fe9cb43b3a969c8567ddef4c7 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:08.888755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:08.890016 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d15b5b-1b75-477b-99b5-684d0bd704ca None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:08.897442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b843d31-248c-4e29-bf19-294ff005ab65 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:08.939831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb6f3b-1811-4f9c-9a91-ada44c6cfb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:08.940536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb6f3b-1811-4f9c-9a91-ada44c6cfb62 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:08.940720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb6f3b-1811-4f9c-9a91-ada44c6cfb62 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:08.940998 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb6f3b-1811-4f9c-9a91-ada44c6cfb62 None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:08.941642 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb6f3b-1811-4f9c-9a91-ada44c6cfb62 None admin] RBAC: Adding query filter params (name=image) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:08.941947 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb6f3b-1811-4f9c-9a91-ada44c6cfb62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:08.943397 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb6f3b-1811-4f9c-9a91-ada44c6cfb62 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:08.948841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 530/1040] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:08.955905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb6f3b-1811-4f9c-9a91-ada44c6cfb62 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:09.026116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d15b5b-1b75-477b-99b5-684d0bd704ca None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:09.048415 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:09.064844 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a7dd6-9f6e-4035-be3f-b05fe50d4c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:09.065621 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a7dd6-9f6e-4035-be3f-b05fe50d4c82 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:09.066015 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a7dd6-9f6e-4035-be3f-b05fe50d4c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:09.066431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a7dd6-9f6e-4035-be3f-b05fe50d4c82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:09.066654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb6f3b-1811-4f9c-9a91-ada44c6cfb62 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:09.067839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a7dd6-9f6e-4035-be3f-b05fe50d4c82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:09.069285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a7dd6-9f6e-4035-be3f-b05fe50d4c82 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:09.075060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 511/1041] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:09.077415 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a7dd6-9f6e-4035-be3f-b05fe50d4c82 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:09.103276 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893064f-df2a-4d6a-846a-a8a4dca93113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:09.104057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893064f-df2a-4d6a-846a-a8a4dca93113 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:09.104347 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893064f-df2a-4d6a-846a-a8a4dca93113 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:09.104933 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893064f-df2a-4d6a-846a-a8a4dca93113 None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:09.105877 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893064f-df2a-4d6a-846a-a8a4dca93113 None admin] RBAC: Adding query filter params (type=image) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:09.106189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893064f-df2a-4d6a-846a-a8a4dca93113 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:09.107587 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893064f-df2a-4d6a-846a-a8a4dca93113 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:09.117842 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 531/1042] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:09.120472 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893064f-df2a-4d6a-846a-a8a4dca93113 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:09.151279 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:09.193439 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a7dd6-9f6e-4035-be3f-b05fe50d4c82 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:09.229113 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ddc3b5-9222-4f89-8c6e-5aa20127fda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:09.229720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddc3b5-9222-4f89-8c6e-5aa20127fda4 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:09.229998 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddc3b5-9222-4f89-8c6e-5aa20127fda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:09.230270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddc3b5-9222-4f89-8c6e-5aa20127fda4 None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:09.230939 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ddc3b5-9222-4f89-8c6e-5aa20127fda4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:09.232374 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ddc3b5-9222-4f89-8c6e-5aa20127fda4 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:09.237995 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893064f-df2a-4d6a-846a-a8a4dca93113 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:09.258376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 512/1043] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:46:09 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:46:09.292236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097091e3-e8f4-4385-a410-af1336b4eccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:09.292870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097091e3-e8f4-4385-a410-af1336b4eccc None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:09.293156 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097091e3-e8f4-4385-a410-af1336b4eccc None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:09.293428 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097091e3-e8f4-4385-a410-af1336b4eccc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:09.294154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097091e3-e8f4-4385-a410-af1336b4eccc None admin] RBAC: Adding query filter params (interface=public, service_id=8c4fd075f3d8404f8cfeb0da20935d5d, region_id=RegionOne) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:09.294468 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097091e3-e8f4-4385-a410-af1336b4eccc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:09.296463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097091e3-e8f4-4385-a410-af1336b4eccc None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:09.302452 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 532/1044] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Sep 22 14:46:09 2022] GET /identity/v3/endpoints?service_id=8c4fd075f3d8404f8cfeb0da20935d5d&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:09.308610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ddc3b5-9222-4f89-8c6e-5aa20127fda4 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:09.368818 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:09.432529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ddc3b5-9222-4f89-8c6e-5aa20127fda4 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:09.471282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa488e-6276-417c-ad01-ba2bc4670bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:09.472150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa488e-6276-417c-ad01-ba2bc4670bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:09.472434 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa488e-6276-417c-ad01-ba2bc4670bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:09.472702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa488e-6276-417c-ad01-ba2bc4670bb8 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:09.473313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa488e-6276-417c-ad01-ba2bc4670bb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:09.474677 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa488e-6276-417c-ad01-ba2bc4670bb8 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:09.481774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 513/1045] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Sep 22 14:46:09 2022] GET /identity/v3/services => generated 1957 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:46:10.972031 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b0a41-a0b8-4927-a538-ef9f0fe8c91d None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:10.972622 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b0a41-a0b8-4927-a538-ef9f0fe8c91d None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:10.972622 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b0a41-a0b8-4927-a538-ef9f0fe8c91d None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:10.973956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 533/1046] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:10.979214 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9615b-2322-4501-be26-ffdcf950a02a None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:10.979690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9615b-2322-4501-be26-ffdcf950a02a None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:10.979772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9615b-2322-4501-be26-ffdcf950a02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:10.998506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-3ea9615b-2322-4501-be26-ffdcf950a02a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:11.000169 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3ea9615b-2322-4501-be26-ffdcf950a02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:11.072542 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 514/1047] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:11.350905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b0a41-a0b8-4927-a538-ef9f0fe8c91d None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:11.445369 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:11.459834 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b0a41-a0b8-4927-a538-ef9f0fe8c91d None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:11.503541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a522280-6140-48f2-b406-ba05c6fad1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:11.504181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a522280-6140-48f2-b406-ba05c6fad1c9 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:11.504461 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a522280-6140-48f2-b406-ba05c6fad1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:11.504734 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a522280-6140-48f2-b406-ba05c6fad1c9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:11.512643 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b55d5-8e8a-4c97-a18e-9b3545a67a16 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:11.512930 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b55d5-8e8a-4c97-a18e-9b3545a67a16 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:11.513203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b55d5-8e8a-4c97-a18e-9b3545a67a16 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:11.514716 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 515/1048] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:11.516652 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a522280-6140-48f2-b406-ba05c6fad1c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:11.519660 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b85d8-288a-4493-833e-2c93300f1f21 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:11.519946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b85d8-288a-4493-833e-2c93300f1f21 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:11.520216 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b85d8-288a-4493-833e-2c93300f1f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:11.521609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-5a522280-6140-48f2-b406-ba05c6fad1c9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:46:11.521609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:11.521609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:11.521609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:11.521609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:11.521609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:11.521609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:11.521609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:11.521609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:11.521609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:11.521609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:11.521609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:11.521609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:11.521609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:11.521609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:11.521609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:11.522588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:11.522588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:11.522588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:11.522588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:11.522588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:11.522588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:11.522588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:11.522588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:11.522588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:11.522588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:11.522588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:11.522588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:11.522588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:11.522588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:11.522588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:11.522588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:11.527898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:11.527898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:11.527898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:11.527898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:11.527898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:11.527898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:11.527898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:11.527898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:11.527898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:11.527898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:11.527898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:46:11.527898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:11.527898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 534/1049] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Sep 22 14:46:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:11.528661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a522280-6140-48f2-b406-ba05c6fad1c9 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:11.547788 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-918b85d8-288a-4493-833e-2c93300f1f21 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:11.549587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-918b85d8-288a-4493-833e-2c93300f1f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:11.584504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:11.632870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 516/1050] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:46:11.651692 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a522280-6140-48f2-b406-ba05c6fad1c9 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:11.687198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12984905-d82b-4378-a108-a1cc02cea858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:11.687795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12984905-d82b-4378-a108-a1cc02cea858 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:11.688068 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12984905-d82b-4378-a108-a1cc02cea858 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:11.688336 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12984905-d82b-4378-a108-a1cc02cea858 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:11.688980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12984905-d82b-4378-a108-a1cc02cea858 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:11.689280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12984905-d82b-4378-a108-a1cc02cea858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:11.690624 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12984905-d82b-4378-a108-a1cc02cea858 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:11.693781 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518c3c9-5588-4862-802d-ffcc9b914d72 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:11.694074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518c3c9-5588-4862-802d-ffcc9b914d72 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:11.694370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518c3c9-5588-4862-802d-ffcc9b914d72 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:11.695887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 517/1051] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:46:11.704466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 535/1052] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Sep 22 14:46:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:11.704634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3f03b-760a-4886-b38c-b595e6582a28 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:11.704634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3f03b-760a-4886-b38c-b595e6582a28 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:11.704634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3f03b-760a-4886-b38c-b595e6582a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:11.705306 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12984905-d82b-4378-a108-a1cc02cea858 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:11.719420 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:11.720636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-84d3f03b-760a-4886-b38c-b595e6582a28 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:11.722345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-84d3f03b-760a-4886-b38c-b595e6582a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:11.797085 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 518/1053] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:46:11.816441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12984905-d82b-4378-a108-a1cc02cea858 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:11.850128 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da296e44-7da5-41a9-bb48-20e7ea0d2732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:11.850674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da296e44-7da5-41a9-bb48-20e7ea0d2732 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:11.850971 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da296e44-7da5-41a9-bb48-20e7ea0d2732 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:11.851241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da296e44-7da5-41a9-bb48-20e7ea0d2732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:11.861677 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da296e44-7da5-41a9-bb48-20e7ea0d2732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:11.863129 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da296e44-7da5-41a9-bb48-20e7ea0d2732 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:11.877460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-da296e44-7da5-41a9-bb48-20e7ea0d2732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:11.877460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:11.877460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:11.877460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:11.877460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:11.877460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:11.877460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:11.877460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:11.877460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:11.877460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:11.877460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:11.877460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:11.877460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:46:11.877460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:46:11.877460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:46:11.877460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:46:11.884412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:11.884412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:11.884412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:11.884412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:11.884412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:11.884412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:11.884412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:11.884412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:11.884412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:11.884412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:11.884412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:11.884412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:11.884412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:11.884412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:11.884412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:11.884412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:11.884412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:46:11.885802 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:46:11.885802 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:11.885802 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:11.885802 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 536/1054] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:11.886103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3f03b-760a-4886-b38c-b595e6582a28 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:11.911207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da296e44-7da5-41a9-bb48-20e7ea0d2732 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:11.987852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:12.000748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:12.005453 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3f03b-760a-4886-b38c-b595e6582a28 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:12.031283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da296e44-7da5-41a9-bb48-20e7ea0d2732 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:12.042979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58d4b0-359a-4d9c-8e8a-2c6c3c7d4d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:12.043588 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58d4b0-359a-4d9c-8e8a-2c6c3c7d4d1b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:12.043861 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58d4b0-359a-4d9c-8e8a-2c6c3c7d4d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:12.044127 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58d4b0-359a-4d9c-8e8a-2c6c3c7d4d1b None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:12.044756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58d4b0-359a-4d9c-8e8a-2c6c3c7d4d1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:12.045063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58d4b0-359a-4d9c-8e8a-2c6c3c7d4d1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:12.046978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58d4b0-359a-4d9c-8e8a-2c6c3c7d4d1b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:12.058777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 519/1055] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:12.064111 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58d4b0-359a-4d9c-8e8a-2c6c3c7d4d1b None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:12.075512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43b730-f66f-4ff5-9760-f7f6f2f78125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:12.076566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b730-f66f-4ff5-9760-f7f6f2f78125 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:12.076854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b730-f66f-4ff5-9760-f7f6f2f78125 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:12.077457 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b730-f66f-4ff5-9760-f7f6f2f78125 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:12.089076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43b730-f66f-4ff5-9760-f7f6f2f78125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:12.094448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-8e43b730-f66f-4ff5-9760-f7f6f2f78125 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:12.094448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:12.094448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:12.094448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:12.094448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:12.094448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:12.094448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:12.094448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:12.094448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:12.094448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:12.094448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:12.094448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:12.094448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:12.094448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:12.094448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:12.094448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:12.095480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:12.095480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:12.095480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:12.095480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:12.095480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:12.095480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:12.095480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:12.095480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:12.095480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:12.095480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:12.095480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:12.095480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:12.095480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:12.095480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:12.095480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:12.095480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:12.096332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:12.096332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:12.096332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:12.096332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:12.096332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:12.096332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:12.096332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:12.096332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:12.096332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:12.096332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:12.096332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:12.096332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:12.096332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 537/1056] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:46:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:12.098275 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43b730-f66f-4ff5-9760-f7f6f2f78125 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:12.179440 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58d4b0-359a-4d9c-8e8a-2c6c3c7d4d1b None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:12.200763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43b730-f66f-4ff5-9760-f7f6f2f78125 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:12.213617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c30032-3c94-41f9-80e2-ba186304aee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:12.214248 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c30032-3c94-41f9-80e2-ba186304aee4 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:12.214532 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c30032-3c94-41f9-80e2-ba186304aee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:12.214840 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c30032-3c94-41f9-80e2-ba186304aee4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:12.223034 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c30032-3c94-41f9-80e2-ba186304aee4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:12.223860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:12.224846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c30032-3c94-41f9-80e2-ba186304aee4 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:12.236493 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca8dae-603d-4425-8e19-ac4669853864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:12.237074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-74c30032-3c94-41f9-80e2-ba186304aee4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:46:12.237074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:12.237074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:12.237074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:12.237074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:12.237074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:12.237074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:12.237074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:12.237074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:12.237074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:12.237074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:12.237074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:12.237074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:46:12.237074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:46:12.237074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:46:12.237074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:12.238217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:12.239236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:12.239236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:12.239236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:46:12.239236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:46:12.239236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:46:12.239236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:46:12.239236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:46:12.239236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:46:12.239236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:46:12.239236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:12.239800 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca8dae-603d-4425-8e19-ac4669853864 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:12.239800 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca8dae-603d-4425-8e19-ac4669853864 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:12.239800 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca8dae-603d-4425-8e19-ac4669853864 None admin] PATH_INFO: `/v3/services/image` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:12.239800 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca8dae-603d-4425-8e19-ac4669853864 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:12.240070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 520/1057] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Sep 22 14:46:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:12.240070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c30032-3c94-41f9-80e2-ba186304aee4 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:12.240207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca8dae-603d-4425-8e19-ac4669853864 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:12.250753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-79ca8dae-603d-4425-8e19-ac4669853864 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:12.250753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:12.250753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:46:12.250753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:46:12.250753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:46:12.250753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:46:12.250753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:46:12.250753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:12.250753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:12.250753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:12.250753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:46:12.250753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:12.250753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:12.250753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:12.250753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:12.250753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:12.250753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:12.252016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:12.252016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:12.252016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:12.252016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:12.252016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:12.252016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:12.252016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:46:12.252016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:46:12.252016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:46:12.252016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:46:12.252016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:12.252016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:12.252016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:12.252016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:12.252016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:12.252016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:12.253370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:12.253370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:12.253370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:12.253370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:12.253370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:12.253370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:12.253370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:12.253370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:12.253370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:12.253370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:12.253370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:46:12.253370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:46:12.253370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 14:46:12.253370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:12.253370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 538/1058] 10.222.0.26 () {56 vars in 1105 bytes} [Thu Sep 22 14:46:12 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:12.254402 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca8dae-603d-4425-8e19-ac4669853864 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:12.263844 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:12.347477 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c30032-3c94-41f9-80e2-ba186304aee4 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:12.356888 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca8dae-603d-4425-8e19-ac4669853864 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:12.380532 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d968e-1f26-44bf-aa87-d766ee92264e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:12.381135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d968e-1f26-44bf-aa87-d766ee92264e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:12.381405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d968e-1f26-44bf-aa87-d766ee92264e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:12.381715 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d968e-1f26-44bf-aa87-d766ee92264e None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:12.382349 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d968e-1f26-44bf-aa87-d766ee92264e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:12.382646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d968e-1f26-44bf-aa87-d766ee92264e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:12.384098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d968e-1f26-44bf-aa87-d766ee92264e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:12.391405 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc635e8-781e-422e-807c-7a131943c819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:12.392008 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc635e8-781e-422e-807c-7a131943c819 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:12.392283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc635e8-781e-422e-807c-7a131943c819 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:12.392551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc635e8-781e-422e-807c-7a131943c819 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:12.393136 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 521/1059] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:46:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:12.393278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc635e8-781e-422e-807c-7a131943c819 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:12.393556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc635e8-781e-422e-807c-7a131943c819 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:12.393927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d968e-1f26-44bf-aa87-d766ee92264e None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:12.395071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc635e8-781e-422e-807c-7a131943c819 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:12.417817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 539/1060] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Sep 22 14:46:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:12.417817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc635e8-781e-422e-807c-7a131943c819 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:12.462300 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:12.479958 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:12.499475 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d968e-1f26-44bf-aa87-d766ee92264e None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:12.518548 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc635e8-781e-422e-807c-7a131943c819 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:12.533764 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3e41e-54b7-4286-b85e-be0e6a1736c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:12.534371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3e41e-54b7-4286-b85e-be0e6a1736c1 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:12.534645 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3e41e-54b7-4286-b85e-be0e6a1736c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:12.535021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3e41e-54b7-4286-b85e-be0e6a1736c1 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:12.535748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3e41e-54b7-4286-b85e-be0e6a1736c1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:12.536135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3e41e-54b7-4286-b85e-be0e6a1736c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:12.537502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3e41e-54b7-4286-b85e-be0e6a1736c1 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:12.542242 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 522/1061] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:12.543040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3e41e-54b7-4286-b85e-be0e6a1736c1 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:12.552664 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e296dcf-b73c-4a92-9d43-cbba3432b446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:12.553286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e296dcf-b73c-4a92-9d43-cbba3432b446 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:12.553560 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e296dcf-b73c-4a92-9d43-cbba3432b446 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:12.553826 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e296dcf-b73c-4a92-9d43-cbba3432b446 None admin] PATH_INFO: `/v3/users/f6e04f929fbb4cf387917f9520f6611a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:12.567543 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e296dcf-b73c-4a92-9d43-cbba3432b446 None admin] RBAC: Authorizing `identity:get_user(user_id=f6e04f929fbb4cf387917f9520f6611a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:12.568976 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e296dcf-b73c-4a92-9d43-cbba3432b446 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:12.570665 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 540/1062] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:12 2022] GET /identity/v3/users/f6e04f929fbb4cf387917f9520f6611a => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:12.571407 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e296dcf-b73c-4a92-9d43-cbba3432b446 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:12.643333 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3e41e-54b7-4286-b85e-be0e6a1736c1 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:12.673466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e296dcf-b73c-4a92-9d43-cbba3432b446 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:12.675022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:12.677279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a508f5-a894-4bb4-977b-797e5e345aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:12.677926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a508f5-a894-4bb4-977b-797e5e345aad None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:12.678206 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a508f5-a894-4bb4-977b-797e5e345aad None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:12.678477 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a508f5-a894-4bb4-977b-797e5e345aad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:12.689413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a508f5-a894-4bb4-977b-797e5e345aad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:12.690883 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a508f5-a894-4bb4-977b-797e5e345aad None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:12.693421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:12.706724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-f6a508f5-a894-4bb4-977b-797e5e345aad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:12.706724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:12.706724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:12.706724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:12.706724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:12.706724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:12.706724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:12.706724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:12.706724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:12.706724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:12.706724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:12.706724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:12.706724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:46:12.706724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:46:12.706724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:46:12.706724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:46:12.707823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:12.707823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:12.707823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:12.707823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:12.707823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:12.707823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:12.707823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:12.707823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:12.707823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:12.707823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:12.707823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:12.707823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:12.707823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:12.707823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:12.707823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:12.707823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:12.707823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:46:12.708827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:46:12.708827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:12.708827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:12.708827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 523/1063] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:12.708827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a508f5-a894-4bb4-977b-797e5e345aad None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:12.714199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d2017-5b98-4b07-8bb5-bf6fa0d2a2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:12.714763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d2017-5b98-4b07-8bb5-bf6fa0d2a2db None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:12.715040 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d2017-5b98-4b07-8bb5-bf6fa0d2a2db None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:12.716085 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d2017-5b98-4b07-8bb5-bf6fa0d2a2db None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:12.716085 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d2017-5b98-4b07-8bb5-bf6fa0d2a2db None admin] RBAC: Adding query filter params (type=image) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:12.716289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d2017-5b98-4b07-8bb5-bf6fa0d2a2db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:12.717649 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d2017-5b98-4b07-8bb5-bf6fa0d2a2db None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:12.722903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 541/1064] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:12.723677 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d2017-5b98-4b07-8bb5-bf6fa0d2a2db None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:12.811592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a508f5-a894-4bb4-977b-797e5e345aad None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:12.820971 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d2017-5b98-4b07-8bb5-bf6fa0d2a2db None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:12.845325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b7b34-ecf4-4b60-8d66-6efc88aa84e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:12.845932 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b7b34-ecf4-4b60-8d66-6efc88aa84e0 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:12.846203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b7b34-ecf4-4b60-8d66-6efc88aa84e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:12.846469 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b7b34-ecf4-4b60-8d66-6efc88aa84e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:12.855403 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b54fd3-daf3-4f79-ae45-1b3a8bf82750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:12.856002 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b54fd3-daf3-4f79-ae45-1b3a8bf82750 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:12.856276 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b54fd3-daf3-4f79-ae45-1b3a8bf82750 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:12.856543 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b54fd3-daf3-4f79-ae45-1b3a8bf82750 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:12.857175 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b54fd3-daf3-4f79-ae45-1b3a8bf82750 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:12.857471 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b54fd3-daf3-4f79-ae45-1b3a8bf82750 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:12.858871 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b54fd3-daf3-4f79-ae45-1b3a8bf82750 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:12.874919 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 542/1065] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:12.874919 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b54fd3-daf3-4f79-ae45-1b3a8bf82750 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:12.900198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:12.918178 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:12.925504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b7b34-ecf4-4b60-8d66-6efc88aa84e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:12.926994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b7b34-ecf4-4b60-8d66-6efc88aa84e0 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:12.978892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b54fd3-daf3-4f79-ae45-1b3a8bf82750 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:13.013464 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39527f-1945-4c44-a218-0fcae23cf210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:13.014779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39527f-1945-4c44-a218-0fcae23cf210 None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:13.014919 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39527f-1945-4c44-a218-0fcae23cf210 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:13.015197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39527f-1945-4c44-a218-0fcae23cf210 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:13.015852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39527f-1945-4c44-a218-0fcae23cf210 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:13.017211 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39527f-1945-4c44-a218-0fcae23cf210 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:13.032087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 524/1066] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:12 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:46:13.032938 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b7b34-ecf4-4b60-8d66-6efc88aa84e0 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:13.041164 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 543/1067] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:46:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:46:13.042118 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39527f-1945-4c44-a218-0fcae23cf210 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:13.143093 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:13.151859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:13.158921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b7b34-ecf4-4b60-8d66-6efc88aa84e0 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:13.178222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39527f-1945-4c44-a218-0fcae23cf210 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:13.198722 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9416068-a6bf-4e6e-a5d8-c032d6d6092c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:13.199395 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9416068-a6bf-4e6e-a5d8-c032d6d6092c None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:13.199673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9416068-a6bf-4e6e-a5d8-c032d6d6092c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:13.199942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9416068-a6bf-4e6e-a5d8-c032d6d6092c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:13.207831 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9416068-a6bf-4e6e-a5d8-c032d6d6092c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:13.209387 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9416068-a6bf-4e6e-a5d8-c032d6d6092c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:13.218258 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67262d-a488-4a49-889a-6f2955db9537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:13.218933 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67262d-a488-4a49-889a-6f2955db9537 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:13.219208 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67262d-a488-4a49-889a-6f2955db9537 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:13.219866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67262d-a488-4a49-889a-6f2955db9537 None admin] PATH_INFO: `/v3/projects/5c2eddde651e4e15a5fc9a302408a457` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:13.221351 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67262d-a488-4a49-889a-6f2955db9537 None admin] RBAC: Authorizing `identity:get_project(project_id=5c2eddde651e4e15a5fc9a302408a457)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:13.221974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-c9416068-a6bf-4e6e-a5d8-c032d6d6092c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:46:13.221974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:13.221974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:13.221974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:13.221974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:13.221974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:13.221974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:13.221974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:13.221974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:13.221974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:13.221974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:13.221974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:13.221974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:46:13.221974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:46:13.221974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:46:13.222896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:46:13.222896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:13.222896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:13.222896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:13.222896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:13.222896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:13.222896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:13.222896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:13.222896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:13.222896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:13.222896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:13.222896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:13.222896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:13.222896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:13.222896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:13.222896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:13.222896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:13.223862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:46:13.223862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:46:13.223862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:46:13.223862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:46:13.223862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:46:13.223862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:46:13.223862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:46:13.223862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:13.224325 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67262d-a488-4a49-889a-6f2955db9537 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:13.224398 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 525/1068] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:46:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:13.224465 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 544/1069] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:13 2022] GET /identity/v3/projects/5c2eddde651e4e15a5fc9a302408a457 => generated 311 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:13.228828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9416068-a6bf-4e6e-a5d8-c032d6d6092c None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:13.230236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67262d-a488-4a49-889a-6f2955db9537 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:13.347099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9416068-a6bf-4e6e-a5d8-c032d6d6092c None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:13.353092 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67262d-a488-4a49-889a-6f2955db9537 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:13.383505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4828f6-c602-4531-aa83-c7a90e61b28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:13.384124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4828f6-c602-4531-aa83-c7a90e61b28e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:13.384461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4828f6-c602-4531-aa83-c7a90e61b28e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:13.384836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4828f6-c602-4531-aa83-c7a90e61b28e None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:13.385625 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4828f6-c602-4531-aa83-c7a90e61b28e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:13.386114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4828f6-c602-4531-aa83-c7a90e61b28e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:13.387601 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4828f6-c602-4531-aa83-c7a90e61b28e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:13.387706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44777fc8-e79f-44e8-81c8-875f09d9133f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:13.388278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44777fc8-e79f-44e8-81c8-875f09d9133f None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:13.388553 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44777fc8-e79f-44e8-81c8-875f09d9133f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:13.388827 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44777fc8-e79f-44e8-81c8-875f09d9133f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:13.390061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44777fc8-e79f-44e8-81c8-875f09d9133f None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=5c2eddde651e4e15a5fc9a302408a457, user.id=f6e04f929fbb4cf387917f9520f6611a) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:13.390364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44777fc8-e79f-44e8-81c8-875f09d9133f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:13.391797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44777fc8-e79f-44e8-81c8-875f09d9133f None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:13.407065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 545/1070] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:46:13 2022] GET /identity/v3/role_assignments?user.id=f6e04f929fbb4cf387917f9520f6611a&scope.project.id=5c2eddde651e4e15a5fc9a302408a457&role.id=32c8a81e339a462e973e76bb528ef35a => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:13.411522 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 526/1071] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:46:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:13.417787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44777fc8-e79f-44e8-81c8-875f09d9133f None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:13.431689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:13.530456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44777fc8-e79f-44e8-81c8-875f09d9133f None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:13.568655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1265e-2a69-4165-a032-df785267358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:13.569365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1265e-2a69-4165-a032-df785267358b None admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:13.569668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1265e-2a69-4165-a032-df785267358b None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:13.569943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1265e-2a69-4165-a032-df785267358b None admin] PATH_INFO: `/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/0c9be41c39494e54a14398a1f03c4d69/roles/4314083fe9cb43b3a969c8567ddef4c7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:13.572278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1265e-2a69-4165-a032-df785267358b None admin] RBAC: Authorizing `identity:create_grant(project_id=43bc5871e4bf441b874474bcae4e5385, user_id=0c9be41c39494e54a14398a1f03c4d69, role_id=4314083fe9cb43b3a969c8567ddef4c7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:13.573725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1265e-2a69-4165-a032-df785267358b None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:13.594044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 546/1072] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:46:13 2022] PUT /identity/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/0c9be41c39494e54a14398a1f03c4d69/roles/4314083fe9cb43b3a969c8567ddef4c7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:46:15.284944 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7fcf7-2183-44fd-94ba-e775609f8da7 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:15.285509 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7fcf7-2183-44fd-94ba-e775609f8da7 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:15.285509 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7fcf7-2183-44fd-94ba-e775609f8da7 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:15.287224 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 527/1073] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:15.292413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd705d-1c88-4484-962d-5c28e8b763d7 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:15.292831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd705d-1c88-4484-962d-5c28e8b763d7 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:15.293023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd705d-1c88-4484-962d-5c28e8b763d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:15.315650 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-2afd705d-1c88-4484-962d-5c28e8b763d7 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:15.316860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-2afd705d-1c88-4484-962d-5c28e8b763d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:15.382803 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:15.390870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 547/1074] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:15.549611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830948e5-efa7-4ca8-97af-3cf9d4b013f9 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:15.549784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830948e5-efa7-4ca8-97af-3cf9d4b013f9 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:15.550044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830948e5-efa7-4ca8-97af-3cf9d4b013f9 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:15.551591 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 528/1075] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:15.556545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4bae7-db98-4fbb-a442-19a45466b0d7 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:15.556828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4bae7-db98-4fbb-a442-19a45466b0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:15.557101 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff4bae7-db98-4fbb-a442-19a45466b0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:15.575085 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-fff4bae7-db98-4fbb-a442-19a45466b0d7 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:15.576264 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-fff4bae7-db98-4fbb-a442-19a45466b0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:15.644535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 548/1076] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:15.655632 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830948e5-efa7-4ca8-97af-3cf9d4b013f9 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:15.676318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:15.769109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830948e5-efa7-4ca8-97af-3cf9d4b013f9 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:15.782271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082d45-8cd5-4d41-9ecb-9a891d1fb13f None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:15.782808 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082d45-8cd5-4d41-9ecb-9a891d1fb13f None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:15.783090 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f082d45-8cd5-4d41-9ecb-9a891d1fb13f None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:15.784562 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 549/1077] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:46:15.789792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d8d5c-b8f0-40e5-9d62-3abba4a9e4d3 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:15.790078 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d8d5c-b8f0-40e5-9d62-3abba4a9e4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:15.790352 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d8d5c-b8f0-40e5-9d62-3abba4a9e4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:15.807493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e69175-b25a-4f8d-863b-ba15bb7f9a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:15.808023 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e69175-b25a-4f8d-863b-ba15bb7f9a43 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:15.808297 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e69175-b25a-4f8d-863b-ba15bb7f9a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:15.808564 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e69175-b25a-4f8d-863b-ba15bb7f9a43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:15.810177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-9f6d8d5c-b8f0-40e5-9d62-3abba4a9e4d3 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:15.811906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-9f6d8d5c-b8f0-40e5-9d62-3abba4a9e4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:15.886824 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 550/1078] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:15.887045 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e69175-b25a-4f8d-863b-ba15bb7f9a43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:15.887045 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e69175-b25a-4f8d-863b-ba15bb7f9a43 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:15.910001 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:15.911829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d8d5c-b8f0-40e5-9d62-3abba4a9e4d3 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:15.973497 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:16.015866 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 529/1079] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:46:16.017060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e69175-b25a-4f8d-863b-ba15bb7f9a43 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:16.025574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d8d5c-b8f0-40e5-9d62-3abba4a9e4d3 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:16.065730 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34483b-759d-41c3-a3be-0529025a4f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:16.066484 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34483b-759d-41c3-a3be-0529025a4f67 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:16.066814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34483b-759d-41c3-a3be-0529025a4f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:16.067184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34483b-759d-41c3-a3be-0529025a4f67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:16.079778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34483b-759d-41c3-a3be-0529025a4f67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:16.085942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-de34483b-759d-41c3-a3be-0529025a4f67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:16.085942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:16.085942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:16.085942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:16.085942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:16.085942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:16.085942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:16.085942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:16.085942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:16.085942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:16.085942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:16.085942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:16.085942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:16.085942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:16.085942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:16.085942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:16.086958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:16.086958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:16.086958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:16.086958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:16.086958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:16.086958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:16.086958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:16.086958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:16.086958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:16.086958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:16.086958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:16.086958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:16.086958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:16.086958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:16.086958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:16.086958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:16.087852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:16.087852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:16.087852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:16.087852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:16.087852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:16.087852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:16.087852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:16.087852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:16.087852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:16.087852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:16.087852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:16.087852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:16.087852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 551/1080] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:46:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:16.094520 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34483b-759d-41c3-a3be-0529025a4f67 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:16.133989 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e69175-b25a-4f8d-863b-ba15bb7f9a43 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:16.151587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:16.171502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae0b0b-8d43-490e-ba47-c3c13ae61dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:16.172128 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae0b0b-8d43-490e-ba47-c3c13ae61dba None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:16.172400 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae0b0b-8d43-490e-ba47-c3c13ae61dba None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:16.172665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae0b0b-8d43-490e-ba47-c3c13ae61dba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:16.182619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae0b0b-8d43-490e-ba47-c3c13ae61dba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:16.184102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae0b0b-8d43-490e-ba47-c3c13ae61dba None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:16.199379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34483b-759d-41c3-a3be-0529025a4f67 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:16.199539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-5aae0b0b-8d43-490e-ba47-c3c13ae61dba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:16.199539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:16.199539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:16.199539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:16.199539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:16.199539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:16.199539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:16.199539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:16.199539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:16.199539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:16.199539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:16.199539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:16.199539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:46:16.199539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:46:16.199539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:46:16.199539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:46:16.200513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:16.200513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:16.200513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:16.200513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:16.200513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:16.200513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:16.200513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:16.200513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:16.200513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:16.200513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:16.200513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:16.200513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:16.200513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:16.200513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:16.200513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:16.200513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:16.200513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:46:16.201952 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:46:16.201952 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:16.201952 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:16.201952 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 530/1081] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:16.201952 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae0b0b-8d43-490e-ba47-c3c13ae61dba None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:16.222122 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:16.236432 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d1345-caa7-484e-959d-45b4e7b93a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:16.237039 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d1345-caa7-484e-959d-45b4e7b93a2f None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:16.237320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d1345-caa7-484e-959d-45b4e7b93a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:16.237600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d1345-caa7-484e-959d-45b4e7b93a2f None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:16.238250 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d1345-caa7-484e-959d-45b4e7b93a2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:16.238555 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d1345-caa7-484e-959d-45b4e7b93a2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:16.239953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d1345-caa7-484e-959d-45b4e7b93a2f None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:16.248174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 552/1082] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:46:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:16.251599 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d1345-caa7-484e-959d-45b4e7b93a2f None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:16.302936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae0b0b-8d43-490e-ba47-c3c13ae61dba None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:16.341143 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cf11a-6602-451b-8394-6c90e285eb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:16.341774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cf11a-6602-451b-8394-6c90e285eb77 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:16.342668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cf11a-6602-451b-8394-6c90e285eb77 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:16.342969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cf11a-6602-451b-8394-6c90e285eb77 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:16.353871 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195cf11a-6602-451b-8394-6c90e285eb77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:16.357042 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d1345-caa7-484e-959d-45b4e7b93a2f None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:16.358598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-195cf11a-6602-451b-8394-6c90e285eb77 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:46:16.358598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:16.358598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:16.358598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:16.358598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:16.358598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:16.358598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:16.358598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:16.358598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:16.358598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:16.358598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:16.358598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:16.358598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:16.358598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:16.358598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:16.358598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:16.359612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:16.359612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:16.359612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:16.359612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:16.359612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:16.359612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:16.359612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:16.359612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:16.359612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:16.359612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:16.359612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:16.359612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:16.359612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:16.359612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:16.359612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:16.359612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:16.360491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:16.360491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:16.360491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:16.360491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:16.360491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:16.360491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:16.360491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:16.360491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:16.360491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:16.360491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:16.360491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 14:46:16.360491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:16.360491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 531/1083] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Sep 22 14:46:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:16.360491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cf11a-6602-451b-8394-6c90e285eb77 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:16.395592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:16.400637 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f97999-a135-4d99-963d-7b57056b98d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:16.401242 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f97999-a135-4d99-963d-7b57056b98d9 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:16.401526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f97999-a135-4d99-963d-7b57056b98d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:16.401794 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f97999-a135-4d99-963d-7b57056b98d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:16.402427 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f97999-a135-4d99-963d-7b57056b98d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:16.402747 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f97999-a135-4d99-963d-7b57056b98d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:16.404107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f97999-a135-4d99-963d-7b57056b98d9 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:16.416381 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 553/1084] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:16.417204 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f97999-a135-4d99-963d-7b57056b98d9 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:16.470361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:16.485587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cf11a-6602-451b-8394-6c90e285eb77 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:16.524890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87aad50-cfb6-46c4-9713-f6f0d080b409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:16.525543 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87aad50-cfb6-46c4-9713-f6f0d080b409 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:16.525929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87aad50-cfb6-46c4-9713-f6f0d080b409 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:16.526213 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87aad50-cfb6-46c4-9713-f6f0d080b409 None admin] PATH_INFO: `/v3/users/f6e04f929fbb4cf387917f9520f6611a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:16.527430 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87aad50-cfb6-46c4-9713-f6f0d080b409 None admin] RBAC: Authorizing `identity:get_user(user_id=f6e04f929fbb4cf387917f9520f6611a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:16.528898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87aad50-cfb6-46c4-9713-f6f0d080b409 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:16.535832 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 532/1085] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:16 2022] GET /identity/v3/users/f6e04f929fbb4cf387917f9520f6611a => generated 285 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:16.537858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87aad50-cfb6-46c4-9713-f6f0d080b409 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:16.540211 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f97999-a135-4d99-963d-7b57056b98d9 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:16.586392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5080c95-54df-49f1-a511-e95a98e736fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:16.586603 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5080c95-54df-49f1-a511-e95a98e736fc None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:16.586818 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5080c95-54df-49f1-a511-e95a98e736fc None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:16.587091 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5080c95-54df-49f1-a511-e95a98e736fc None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:16.587762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5080c95-54df-49f1-a511-e95a98e736fc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:16.588060 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5080c95-54df-49f1-a511-e95a98e736fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:16.589511 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5080c95-54df-49f1-a511-e95a98e736fc None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:16.599379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 554/1086] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Sep 22 14:46:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:16.605949 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5080c95-54df-49f1-a511-e95a98e736fc None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:16.652469 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:16.659008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87aad50-cfb6-46c4-9713-f6f0d080b409 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:16.700997 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114129f6-f887-4354-a0a5-c19703f72a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:16.702771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114129f6-f887-4354-a0a5-c19703f72a72 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:16.703620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114129f6-f887-4354-a0a5-c19703f72a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:16.703898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114129f6-f887-4354-a0a5-c19703f72a72 None admin] PATH_INFO: `/v3/projects/5c2eddde651e4e15a5fc9a302408a457` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:16.705450 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114129f6-f887-4354-a0a5-c19703f72a72 None admin] RBAC: Authorizing `identity:get_project(project_id=5c2eddde651e4e15a5fc9a302408a457)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:16.707286 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114129f6-f887-4354-a0a5-c19703f72a72 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:16.709980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 533/1087] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:16 2022] GET /identity/v3/projects/5c2eddde651e4e15a5fc9a302408a457 => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:16.716696 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114129f6-f887-4354-a0a5-c19703f72a72 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:16.730432 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5080c95-54df-49f1-a511-e95a98e736fc None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:16.733213 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:16.771250 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b162ff-ffb7-4de5-b3c2-64b7a475647b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:16.771889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b162ff-ffb7-4de5-b3c2-64b7a475647b None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:16.772168 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b162ff-ffb7-4de5-b3c2-64b7a475647b None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:16.772444 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b162ff-ffb7-4de5-b3c2-64b7a475647b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:16.783890 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b162ff-ffb7-4de5-b3c2-64b7a475647b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:16.785325 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b162ff-ffb7-4de5-b3c2-64b7a475647b None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:16.799953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-94b162ff-ffb7-4de5-b3c2-64b7a475647b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:16.799953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:16.799953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:16.799953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:16.799953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:16.799953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:16.799953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:16.799953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:16.799953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:16.799953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:16.799953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:16.799953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:16.799953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:46:16.799953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:46:16.799953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:46:16.799953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:46:16.801056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:16.801056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:16.801056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:16.801056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:16.801056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:16.801056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:16.801056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:16.801056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:16.801056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:16.801056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:16.801056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:16.801056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:16.801056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:16.801056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:16.801056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:16.801056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:16.801056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:46:16.802233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:46:16.802233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:16.802233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:16.802233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 555/1088] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:16.807014 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b162ff-ffb7-4de5-b3c2-64b7a475647b None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:16.822492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114129f6-f887-4354-a0a5-c19703f72a72 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:16.855467 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b2ca6-a0df-48c5-b0c1-105edd410a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:16.856049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b2ca6-a0df-48c5-b0c1-105edd410a5d None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:16.856322 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b2ca6-a0df-48c5-b0c1-105edd410a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:16.856587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b2ca6-a0df-48c5-b0c1-105edd410a5d None admin] PATH_INFO: `/v3/projects/5c2eddde651e4e15a5fc9a302408a457/users/f6e04f929fbb4cf387917f9520f6611a/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:16.859052 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b2ca6-a0df-48c5-b0c1-105edd410a5d None admin] RBAC: Authorizing `identity:create_grant(project_id=5c2eddde651e4e15a5fc9a302408a457, user_id=f6e04f929fbb4cf387917f9520f6611a, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:16.860466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b2ca6-a0df-48c5-b0c1-105edd410a5d None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:16.874235 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 534/1089] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:46:16 2022] PUT /identity/v3/projects/5c2eddde651e4e15a5fc9a302408a457/users/f6e04f929fbb4cf387917f9520f6611a/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:46:16.913636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b162ff-ffb7-4de5-b3c2-64b7a475647b None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:16.949877 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7598ad-d240-4b66-85d3-3de8e877c9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:16.950510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7598ad-d240-4b66-85d3-3de8e877c9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:16.950793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7598ad-d240-4b66-85d3-3de8e877c9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:16.951089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7598ad-d240-4b66-85d3-3de8e877c9e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:16.951843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7598ad-d240-4b66-85d3-3de8e877c9e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:16.952071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7598ad-d240-4b66-85d3-3de8e877c9e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:16.953681 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7598ad-d240-4b66-85d3-3de8e877c9e8 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:16.967623 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 556/1090] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:46:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:46:16.973736 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b2ca6-a0df-48c5-b0c1-105edd410a5d None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:17.005351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:17.089477 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b2ca6-a0df-48c5-b0c1-105edd410a5d None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:17.126897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1766175-88aa-4fdd-aefc-99630ea72239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:17.127446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1766175-88aa-4fdd-aefc-99630ea72239 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:17.127725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1766175-88aa-4fdd-aefc-99630ea72239 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:17.127993 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1766175-88aa-4fdd-aefc-99630ea72239 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:17.135555 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1766175-88aa-4fdd-aefc-99630ea72239 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:17.136996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1766175-88aa-4fdd-aefc-99630ea72239 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:17.148743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-a1766175-88aa-4fdd-aefc-99630ea72239 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:46:17.148743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:17.148743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:17.148743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:17.148743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:17.148743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:17.148743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:17.148743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:17.148743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:17.148743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:17.148743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:17.148743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:17.148743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:46:17.148743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:46:17.148743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:46:17.148743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:17.149902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:17.150920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:17.150920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:17.150920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:46:17.150920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:46:17.150920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:46:17.150920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:46:17.150920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:46:17.150920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:46:17.150920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 14:46:17.150920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:17.150920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 535/1091] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Sep 22 14:46:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:17.156651 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7598ad-d240-4b66-85d3-3de8e877c9e8 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:17.174530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:17.260907 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7598ad-d240-4b66-85d3-3de8e877c9e8 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:17.294496 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322c0df-df03-4801-93b3-141e2a7f30c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:17.295159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322c0df-df03-4801-93b3-141e2a7f30c6 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:17.295436 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322c0df-df03-4801-93b3-141e2a7f30c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:17.295705 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322c0df-df03-4801-93b3-141e2a7f30c6 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:17.296421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0322c0df-df03-4801-93b3-141e2a7f30c6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:17.296722 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0322c0df-df03-4801-93b3-141e2a7f30c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:17.298131 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0322c0df-df03-4801-93b3-141e2a7f30c6 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:17.316286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 557/1092] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Sep 22 14:46:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:17.322196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1766175-88aa-4fdd-aefc-99630ea72239 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:17.410050 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:17.427087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1766175-88aa-4fdd-aefc-99630ea72239 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:17.460157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31417d-883e-4441-9c2d-3aa0dfab905e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:17.460765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31417d-883e-4441-9c2d-3aa0dfab905e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:17.461045 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31417d-883e-4441-9c2d-3aa0dfab905e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:17.461319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31417d-883e-4441-9c2d-3aa0dfab905e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:17.529009 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31417d-883e-4441-9c2d-3aa0dfab905e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:17.531072 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31417d-883e-4441-9c2d-3aa0dfab905e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:17.613126 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:17.630331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 536/1093] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 14:46:17.636207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322c0df-df03-4801-93b3-141e2a7f30c6 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:17.710241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:17.735086 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322c0df-df03-4801-93b3-141e2a7f30c6 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:17.771222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc720bf3-f625-4afc-bcbc-60dc84c9d54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:17.771863 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc720bf3-f625-4afc-bcbc-60dc84c9d54f None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:17.772139 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc720bf3-f625-4afc-bcbc-60dc84c9d54f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:17.772412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc720bf3-f625-4afc-bcbc-60dc84c9d54f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:17.783332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc720bf3-f625-4afc-bcbc-60dc84c9d54f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:17.784818 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc720bf3-f625-4afc-bcbc-60dc84c9d54f None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:17.799456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-bc720bf3-f625-4afc-bcbc-60dc84c9d54f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:17.799456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:17.799456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:17.799456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:17.799456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:17.799456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:17.799456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:17.799456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:17.799456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:17.799456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:17.799456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:17.799456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:17.799456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:46:17.799456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:46:17.799456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:46:17.799456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:46:17.800433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:17.800433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:17.800433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:17.800433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:17.800433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:17.800433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:17.800433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:17.800433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:17.800433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:17.800433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:17.800433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:17.800433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:17.800433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:17.800433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:17.800433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:17.800433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:17.800433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:46:17.801369 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:46:17.801369 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:46:17.801369 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:17.801369 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 558/1094] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:17.805877 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31417d-883e-4441-9c2d-3aa0dfab905e None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:17.909093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31417d-883e-4441-9c2d-3aa0dfab905e None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:17.943104 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c7f27-b43b-4078-bf0f-45b3732426ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:17.943702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c7f27-b43b-4078-bf0f-45b3732426ee None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:17.943980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c7f27-b43b-4078-bf0f-45b3732426ee None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:17.944252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c7f27-b43b-4078-bf0f-45b3732426ee None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:17.944877 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c7f27-b43b-4078-bf0f-45b3732426ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:17.945179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c7f27-b43b-4078-bf0f-45b3732426ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:17.946538 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c7f27-b43b-4078-bf0f-45b3732426ee None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:17.959228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 537/1095] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:17.964880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc720bf3-f625-4afc-bcbc-60dc84c9d54f None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:18.068622 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc720bf3-f625-4afc-bcbc-60dc84c9d54f None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:18.094645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:18.104367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7643233-5b57-4586-b384-3d2812f8e9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:18.105007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7643233-5b57-4586-b384-3d2812f8e9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:18.105283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7643233-5b57-4586-b384-3d2812f8e9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:18.105553 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7643233-5b57-4586-b384-3d2812f8e9f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:18.112995 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7643233-5b57-4586-b384-3d2812f8e9f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:18.114435 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7643233-5b57-4586-b384-3d2812f8e9f9 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:18.126590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-a7643233-5b57-4586-b384-3d2812f8e9f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:46:18.126590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:18.126590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:18.126590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:18.126590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:18.126590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:18.126590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:18.126590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:18.126590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:18.126590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:18.126590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:18.126590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:18.126590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:46:18.126590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:46:18.126590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:46:18.127617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:46:18.127617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:18.127617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:18.127617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:18.127617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:18.127617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:18.127617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:18.127617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:18.127617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:18.127617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:18.127617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:18.127617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:18.127617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:18.127617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:18.127617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:18.127617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:18.127617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:18.128556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:46:18.128556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:46:18.128556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:46:18.128556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:46:18.128556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:46:18.128556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:46:18.128556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:46:18.128556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:18.128556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 559/1096] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:46:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:46:18.133580 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c7f27-b43b-4078-bf0f-45b3732426ee None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:18.178746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:18.231594 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c7f27-b43b-4078-bf0f-45b3732426ee None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:18.264540 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b4c7b-747d-43c2-8408-7e247c88c739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:18.265138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b4c7b-747d-43c2-8408-7e247c88c739 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:18.265417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b4c7b-747d-43c2-8408-7e247c88c739 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:18.265684 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b4c7b-747d-43c2-8408-7e247c88c739 None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:18.266363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b4c7b-747d-43c2-8408-7e247c88c739 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:18.266663 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b4c7b-747d-43c2-8408-7e247c88c739 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:18.268113 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b4c7b-747d-43c2-8408-7e247c88c739 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:18.280228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 538/1097] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:46:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:18.286288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7643233-5b57-4586-b384-3d2812f8e9f9 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:18.387400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7643233-5b57-4586-b384-3d2812f8e9f9 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:18.420809 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a96d0d-ec49-4c30-b7cc-4f74c803abbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:18.421411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a96d0d-ec49-4c30-b7cc-4f74c803abbb None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:18.421689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a96d0d-ec49-4c30-b7cc-4f74c803abbb None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:18.421960 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a96d0d-ec49-4c30-b7cc-4f74c803abbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:18.422680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a96d0d-ec49-4c30-b7cc-4f74c803abbb None admin] RBAC: Adding query filter params (role.id=4314083fe9cb43b3a969c8567ddef4c7, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=0c9be41c39494e54a14398a1f03c4d69) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:18.423016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a96d0d-ec49-4c30-b7cc-4f74c803abbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:18.424436 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a96d0d-ec49-4c30-b7cc-4f74c803abbb None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:18.430293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 560/1098] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:46:18 2022] GET /identity/v3/role_assignments?user.id=0c9be41c39494e54a14398a1f03c4d69&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=4314083fe9cb43b3a969c8567ddef4c7 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:19.050027 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ca733-1fcf-47ef-bf76-f57b4be1ace8 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:19.050577 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ca733-1fcf-47ef-bf76-f57b4be1ace8 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:19.050577 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ca733-1fcf-47ef-bf76-f57b4be1ace8 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:19.051949 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 539/1099] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:19.057525 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a95fd-914b-489b-8096-d84e938f2add None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:19.057982 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a95fd-914b-489b-8096-d84e938f2add None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:19.058071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a95fd-914b-489b-8096-d84e938f2add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:19.076770 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-de7a95fd-914b-489b-8096-d84e938f2add None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:19.077971 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-de7a95fd-914b-489b-8096-d84e938f2add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:19.131296 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:19.154124 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 561/1100] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:19.439214 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ca733-1fcf-47ef-bf76-f57b4be1ace8 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:19.537912 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ca733-1fcf-47ef-bf76-f57b4be1ace8 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:19.540197 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:19.571981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e488056-98d9-49e1-89ed-1e26657e9da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:19.572613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e488056-98d9-49e1-89ed-1e26657e9da5 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:19.572886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e488056-98d9-49e1-89ed-1e26657e9da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:19.573157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e488056-98d9-49e1-89ed-1e26657e9da5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:19.583066 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e488056-98d9-49e1-89ed-1e26657e9da5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:19.587894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-5e488056-98d9-49e1-89ed-1e26657e9da5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:19.587894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:19.587894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:19.587894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:19.587894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:19.587894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:19.587894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:19.587894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:19.587894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:19.587894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:19.587894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:19.587894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:19.587894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:19.587894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:19.587894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:19.587894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:19.588979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:19.588979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:19.588979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:19.588979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:19.588979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:19.588979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:19.588979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:19.588979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:19.588979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:19.588979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:19.588979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:19.588979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:19.588979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:19.588979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:19.588979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:19.588979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:19.589985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:19.589985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:19.589985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:19.589985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:19.589985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:19.589985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:19.589985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:19.589985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:19.589985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:19.589985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:19.589985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:19.589985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:19.589985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 540/1101] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:46:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:19.595612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a95fd-914b-489b-8096-d84e938f2add None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:19.692835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a95fd-914b-489b-8096-d84e938f2add None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:19.725819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98f240-4fa4-45e4-9aa5-c54de1727d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:19.726415 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98f240-4fa4-45e4-9aa5-c54de1727d13 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:19.726689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98f240-4fa4-45e4-9aa5-c54de1727d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:19.726990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98f240-4fa4-45e4-9aa5-c54de1727d13 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:19.727641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98f240-4fa4-45e4-9aa5-c54de1727d13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:19.727939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98f240-4fa4-45e4-9aa5-c54de1727d13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:19.729291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98f240-4fa4-45e4-9aa5-c54de1727d13 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:19.737615 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 562/1102] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:46:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:19.743568 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e488056-98d9-49e1-89ed-1e26657e9da5 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:19.840209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e488056-98d9-49e1-89ed-1e26657e9da5 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:19.874013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6776cf-d6d9-4f45-84da-bb4b6bc838cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:19.874630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6776cf-d6d9-4f45-84da-bb4b6bc838cf None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:19.874934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6776cf-d6d9-4f45-84da-bb4b6bc838cf None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:19.875203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6776cf-d6d9-4f45-84da-bb4b6bc838cf None admin] PATH_INFO: `/v3/users/f6e04f929fbb4cf387917f9520f6611a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:19.876332 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6776cf-d6d9-4f45-84da-bb4b6bc838cf None admin] RBAC: Authorizing `identity:get_user(user_id=f6e04f929fbb4cf387917f9520f6611a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:19.877759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6776cf-d6d9-4f45-84da-bb4b6bc838cf None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:19.879372 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 541/1103] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:19 2022] GET /identity/v3/users/f6e04f929fbb4cf387917f9520f6611a => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:19.885273 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98f240-4fa4-45e4-9aa5-c54de1727d13 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:19.927835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:19.984630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98f240-4fa4-45e4-9aa5-c54de1727d13 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:20.016832 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3ed25-bbbe-44be-96e6-25f6621ff47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:20.017430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3ed25-bbbe-44be-96e6-25f6621ff47b None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:20.017706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3ed25-bbbe-44be-96e6-25f6621ff47b None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:20.017976 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3ed25-bbbe-44be-96e6-25f6621ff47b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:20.085473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3ed25-bbbe-44be-96e6-25f6621ff47b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:20.086942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3ed25-bbbe-44be-96e6-25f6621ff47b None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:20.128072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:20.183023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 563/1104] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:46:20.189489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6776cf-d6d9-4f45-84da-bb4b6bc838cf None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:20.220193 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:20.288484 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6776cf-d6d9-4f45-84da-bb4b6bc838cf None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:20.322051 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fcec1-5c71-41d3-9450-d7d571e48260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:20.322679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fcec1-5c71-41d3-9450-d7d571e48260 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:20.322982 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fcec1-5c71-41d3-9450-d7d571e48260 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:20.323252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fcec1-5c71-41d3-9450-d7d571e48260 None admin] PATH_INFO: `/v3/projects/5c2eddde651e4e15a5fc9a302408a457` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:20.324370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fcec1-5c71-41d3-9450-d7d571e48260 None admin] RBAC: Authorizing `identity:get_project(project_id=5c2eddde651e4e15a5fc9a302408a457)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:20.325798 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fcec1-5c71-41d3-9450-d7d571e48260 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:20.328012 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 542/1105] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:20 2022] GET /identity/v3/projects/5c2eddde651e4e15a5fc9a302408a457 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:20.334124 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3ed25-bbbe-44be-96e6-25f6621ff47b None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:20.430984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3ed25-bbbe-44be-96e6-25f6621ff47b None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:20.463070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0acbca-9486-4dc9-b3d6-7fe34f799336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:20.463663 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0acbca-9486-4dc9-b3d6-7fe34f799336 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:20.463940 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0acbca-9486-4dc9-b3d6-7fe34f799336 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:20.464211 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0acbca-9486-4dc9-b3d6-7fe34f799336 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:20.464934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0acbca-9486-4dc9-b3d6-7fe34f799336 None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=5c2eddde651e4e15a5fc9a302408a457, user.id=f6e04f929fbb4cf387917f9520f6611a) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:20.465241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0acbca-9486-4dc9-b3d6-7fe34f799336 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:20.466653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0acbca-9486-4dc9-b3d6-7fe34f799336 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:20.473081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 564/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:46:20 2022] GET /identity/v3/role_assignments?user.id=f6e04f929fbb4cf387917f9520f6611a&scope.project.id=5c2eddde651e4e15a5fc9a302408a457&role.id=32c8a81e339a462e973e76bb528ef35a => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:22.640870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54947d2-c732-4bef-bcdc-d66802d21cb0 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:22.641370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54947d2-c732-4bef-bcdc-d66802d21cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:22.641370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54947d2-c732-4bef-bcdc-d66802d21cb0 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:22.642666 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 543/1107] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:22.648007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4b0ed-141a-4097-9899-d92a36c29227 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:22.648409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4b0ed-141a-4097-9899-d92a36c29227 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:22.648545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4b0ed-141a-4097-9899-d92a36c29227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:22.661905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:22.667471 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-7cf4b0ed-141a-4097-9899-d92a36c29227 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:22.668657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7cf4b0ed-141a-4097-9899-d92a36c29227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:22.734073 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 565/1108] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:23.000233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54947d2-c732-4bef-bcdc-d66802d21cb0 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:23.095134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:23.099683 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54947d2-c732-4bef-bcdc-d66802d21cb0 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:23.132650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bffdc56-11a7-49fb-b0f9-9e32afc7ecce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:23.133280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffdc56-11a7-49fb-b0f9-9e32afc7ecce None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:23.133558 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffdc56-11a7-49fb-b0f9-9e32afc7ecce None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:23.133829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bffdc56-11a7-49fb-b0f9-9e32afc7ecce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:23.134963 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffdc56-11a7-49fb-b0f9-9e32afc7ecce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:23.136399 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bffdc56-11a7-49fb-b0f9-9e32afc7ecce None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:23.137969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 544/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:23.144529 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4b0ed-141a-4097-9899-d92a36c29227 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:23.240979 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4b0ed-141a-4097-9899-d92a36c29227 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:23.274724 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d8595-4b5c-4153-8ac6-96ada6f3b206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:23.275283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d8595-4b5c-4153-8ac6-96ada6f3b206 None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:23.275561 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d8595-4b5c-4153-8ac6-96ada6f3b206 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:23.275831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d8595-4b5c-4153-8ac6-96ada6f3b206 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:23.276471 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d8595-4b5c-4153-8ac6-96ada6f3b206 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:23.277886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d8595-4b5c-4153-8ac6-96ada6f3b206 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:23.287123 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:23.299703 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 566/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:46:23 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:46:25.490577 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049c6a0-a415-4464-a6c9-f2ea1e345549 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:25.491173 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049c6a0-a415-4464-a6c9-f2ea1e345549 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:25.491173 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049c6a0-a415-4464-a6c9-f2ea1e345549 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:25.492494 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 545/1111] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:25.497764 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911f6a9-71f5-47f5-95da-ef7013b912fa None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:25.498659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911f6a9-71f5-47f5-95da-ef7013b912fa None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:25.498888 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911f6a9-71f5-47f5-95da-ef7013b912fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:25.517206 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-c911f6a9-71f5-47f5-95da-ef7013b912fa None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:25.518382 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-c911f6a9-71f5-47f5-95da-ef7013b912fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:25.583239 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 567/1112] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:25.850045 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049c6a0-a415-4464-a6c9-f2ea1e345549 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:25.949659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049c6a0-a415-4464-a6c9-f2ea1e345549 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:25.988227 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a11201-e422-489e-8e3c-b3b23d73f9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:25.988946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a11201-e422-489e-8e3c-b3b23d73f9a5 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:25.989246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a11201-e422-489e-8e3c-b3b23d73f9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:25.989514 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a11201-e422-489e-8e3c-b3b23d73f9a5 None admin] PATH_INFO: `/v3/roles/1b632105fdba460b809a2ecd1d186409` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:25.990960 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a11201-e422-489e-8e3c-b3b23d73f9a5 None admin] RBAC: Authorizing `identity:get_role(role_id=1b632105fdba460b809a2ecd1d186409)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:25.992328 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a11201-e422-489e-8e3c-b3b23d73f9a5 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:25.993663 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 546/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:25 2022] GET /identity/v3/roles/1b632105fdba460b809a2ecd1d186409 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:26.000358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911f6a9-71f5-47f5-95da-ef7013b912fa None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:26.099930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911f6a9-71f5-47f5-95da-ef7013b912fa None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:26.121792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:26.134417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32537285-12bb-4402-a6b0-3d14fd42e87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:26.135077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32537285-12bb-4402-a6b0-3d14fd42e87f None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:26.135356 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32537285-12bb-4402-a6b0-3d14fd42e87f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:26.135626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32537285-12bb-4402-a6b0-3d14fd42e87f None admin] PATH_INFO: `/v3/users/0ae2743864be426da68c42cb43ca4e1f` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:26.148883 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32537285-12bb-4402-a6b0-3d14fd42e87f None admin] RBAC: Authorizing `identity:get_user(user_id=0ae2743864be426da68c42cb43ca4e1f)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:26.150332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32537285-12bb-4402-a6b0-3d14fd42e87f None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:26.151977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 568/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:25 2022] GET /identity/v3/users/0ae2743864be426da68c42cb43ca4e1f => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:26.158497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a11201-e422-489e-8e3c-b3b23d73f9a5 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:26.192709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:26.256400 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a11201-e422-489e-8e3c-b3b23d73f9a5 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:26.289377 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9bc11-4d71-442a-96a6-834daddec679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:26.289970 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9bc11-4d71-442a-96a6-834daddec679 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:26.290246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9bc11-4d71-442a-96a6-834daddec679 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:26.290515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9bc11-4d71-442a-96a6-834daddec679 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:26.370677 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9bc11-4d71-442a-96a6-834daddec679 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:26.373631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9bc11-4d71-442a-96a6-834daddec679 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:26.409426 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:26.474264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 547/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:46:26.480317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32537285-12bb-4402-a6b0-3d14fd42e87f None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:26.578022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32537285-12bb-4402-a6b0-3d14fd42e87f None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:26.612459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328ad10-48a7-4000-acd8-6fa6e7e2796a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:26.613082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328ad10-48a7-4000-acd8-6fa6e7e2796a None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:26.613362 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328ad10-48a7-4000-acd8-6fa6e7e2796a None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:26.613634 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328ad10-48a7-4000-acd8-6fa6e7e2796a None admin] PATH_INFO: `/v3/projects/5c2eddde651e4e15a5fc9a302408a457` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:26.614767 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328ad10-48a7-4000-acd8-6fa6e7e2796a None admin] RBAC: Authorizing `identity:get_project(project_id=5c2eddde651e4e15a5fc9a302408a457)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:26.616261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328ad10-48a7-4000-acd8-6fa6e7e2796a None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:26.617884 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 569/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:26 2022] GET /identity/v3/projects/5c2eddde651e4e15a5fc9a302408a457 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:26.624890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9bc11-4d71-442a-96a6-834daddec679 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:26.726172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9bc11-4d71-442a-96a6-834daddec679 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:26.760676 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e0ff2-f9e3-445a-a534-9ad3b9ea14c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:26.761276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e0ff2-f9e3-445a-a534-9ad3b9ea14c7 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:26.761555 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e0ff2-f9e3-445a-a534-9ad3b9ea14c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:26.761823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e0ff2-f9e3-445a-a534-9ad3b9ea14c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:26.762541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e0ff2-f9e3-445a-a534-9ad3b9ea14c7 None admin] RBAC: Adding query filter params (role.id=1b632105fdba460b809a2ecd1d186409, scope.project.id=5c2eddde651e4e15a5fc9a302408a457, user.id=0ae2743864be426da68c42cb43ca4e1f) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:26.762882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e0ff2-f9e3-445a-a534-9ad3b9ea14c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:26.764335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e0ff2-f9e3-445a-a534-9ad3b9ea14c7 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:26.767111 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:26.770504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 548/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:46:26 2022] GET /identity/v3/role_assignments?user.id=0ae2743864be426da68c42cb43ca4e1f&scope.project.id=5c2eddde651e4e15a5fc9a302408a457&role.id=1b632105fdba460b809a2ecd1d186409 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:28.917277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59284c-e227-4927-8cee-a126a5cb09e3 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:28.917804 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59284c-e227-4927-8cee-a126a5cb09e3 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:28.917804 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59284c-e227-4927-8cee-a126a5cb09e3 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:28.919182 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 570/1118] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:28.924919 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fae793-b5ff-49ea-bce9-ef8cbe795306 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:28.925359 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fae793-b5ff-49ea-bce9-ef8cbe795306 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:28.925452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fae793-b5ff-49ea-bce9-ef8cbe795306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:28.943930 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-a5fae793-b5ff-49ea-bce9-ef8cbe795306 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:28.945109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a5fae793-b5ff-49ea-bce9-ef8cbe795306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:29.011785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 549/1119] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:29.319554 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59284c-e227-4927-8cee-a126a5cb09e3 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:29.369542 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:29.421427 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59284c-e227-4927-8cee-a126a5cb09e3 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:29.456565 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2cfb5-0d00-49fe-813f-3555b19310df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:29.457190 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2cfb5-0d00-49fe-813f-3555b19310df None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:29.457468 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2cfb5-0d00-49fe-813f-3555b19310df None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:29.457734 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2cfb5-0d00-49fe-813f-3555b19310df None admin] PATH_INFO: `/v3/roles/1b632105fdba460b809a2ecd1d186409` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:29.459151 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2cfb5-0d00-49fe-813f-3555b19310df None admin] RBAC: Authorizing `identity:get_role(role_id=1b632105fdba460b809a2ecd1d186409)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:29.460510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2cfb5-0d00-49fe-813f-3555b19310df None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:29.461923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 571/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:29 2022] GET /identity/v3/roles/1b632105fdba460b809a2ecd1d186409 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:29.468274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fae793-b5ff-49ea-bce9-ef8cbe795306 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:29.564593 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fae793-b5ff-49ea-bce9-ef8cbe795306 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:29.593047 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:29.599463 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4be3d-3fa5-46ca-b219-15d342180773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:29.600093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4be3d-3fa5-46ca-b219-15d342180773 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:29.600366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4be3d-3fa5-46ca-b219-15d342180773 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:29.600636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4be3d-3fa5-46ca-b219-15d342180773 None admin] PATH_INFO: `/v3/users/0ae2743864be426da68c42cb43ca4e1f` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:29.601749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d4be3d-3fa5-46ca-b219-15d342180773 None admin] RBAC: Authorizing `identity:get_user(user_id=0ae2743864be426da68c42cb43ca4e1f)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:29.603216 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d4be3d-3fa5-46ca-b219-15d342180773 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:29.604802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 550/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:29 2022] GET /identity/v3/users/0ae2743864be426da68c42cb43ca4e1f => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:29.610637 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2cfb5-0d00-49fe-813f-3555b19310df None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:29.706325 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2cfb5-0d00-49fe-813f-3555b19310df None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:29.723008 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:29.740220 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e374a-4b90-4c43-b3d3-3da3c6656a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:29.740835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e374a-4b90-4c43-b3d3-3da3c6656a49 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:29.741108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e374a-4b90-4c43-b3d3-3da3c6656a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:29.741373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e374a-4b90-4c43-b3d3-3da3c6656a49 None admin] PATH_INFO: `/v3/projects/5c2eddde651e4e15a5fc9a302408a457` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:29.742466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e374a-4b90-4c43-b3d3-3da3c6656a49 None admin] RBAC: Authorizing `identity:get_project(project_id=5c2eddde651e4e15a5fc9a302408a457)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:29.743913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e374a-4b90-4c43-b3d3-3da3c6656a49 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:29.745517 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 572/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:29 2022] GET /identity/v3/projects/5c2eddde651e4e15a5fc9a302408a457 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:29.751491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4be3d-3fa5-46ca-b219-15d342180773 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:29.849984 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4be3d-3fa5-46ca-b219-15d342180773 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:29.883085 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864ac87-d7f1-4787-b2bf-099e71845370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:29.883750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864ac87-d7f1-4787-b2bf-099e71845370 None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:29.884029 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864ac87-d7f1-4787-b2bf-099e71845370 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:29.884303 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864ac87-d7f1-4787-b2bf-099e71845370 None admin] PATH_INFO: `/v3/projects/5c2eddde651e4e15a5fc9a302408a457/users/0ae2743864be426da68c42cb43ca4e1f/roles/1b632105fdba460b809a2ecd1d186409` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:29.886460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864ac87-d7f1-4787-b2bf-099e71845370 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c2eddde651e4e15a5fc9a302408a457, user_id=0ae2743864be426da68c42cb43ca4e1f, role_id=1b632105fdba460b809a2ecd1d186409)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:29.887926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864ac87-d7f1-4787-b2bf-099e71845370 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:29.901118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 551/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:46:29 2022] PUT /identity/v3/projects/5c2eddde651e4e15a5fc9a302408a457/users/0ae2743864be426da68c42cb43ca4e1f/roles/1b632105fdba460b809a2ecd1d186409 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:46:32.013131 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f8a5e-77e5-4770-9044-49de508f1f45 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:32.013686 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f8a5e-77e5-4770-9044-49de508f1f45 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:32.013810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f8a5e-77e5-4770-9044-49de508f1f45 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:32.015079 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 573/1124] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:32.020238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8315d-1c49-4f1d-bd68-2264d4cdb4ec None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:32.020627 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8315d-1c49-4f1d-bd68-2264d4cdb4ec None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:32.020796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8315d-1c49-4f1d-bd68-2264d4cdb4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:32.038605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-72d8315d-1c49-4f1d-bd68-2264d4cdb4ec None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:32.039829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-72d8315d-1c49-4f1d-bd68-2264d4cdb4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:32.091496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:32.106350 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 552/1125] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:32.374577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f8a5e-77e5-4770-9044-49de508f1f45 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:32.474212 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f8a5e-77e5-4770-9044-49de508f1f45 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:32.507631 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d388f19-a871-47d5-8eb7-dfaff8c5b591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:32.508256 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d388f19-a871-47d5-8eb7-dfaff8c5b591 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:32.508538 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d388f19-a871-47d5-8eb7-dfaff8c5b591 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:32.508809 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d388f19-a871-47d5-8eb7-dfaff8c5b591 None admin] PATH_INFO: `/v3/roles/1b632105fdba460b809a2ecd1d186409` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:32.510193 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d388f19-a871-47d5-8eb7-dfaff8c5b591 None admin] RBAC: Authorizing `identity:get_role(role_id=1b632105fdba460b809a2ecd1d186409)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:32.511585 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d388f19-a871-47d5-8eb7-dfaff8c5b591 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:32.512981 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 574/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:32 2022] GET /identity/v3/roles/1b632105fdba460b809a2ecd1d186409 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:32.518870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8315d-1c49-4f1d-bd68-2264d4cdb4ec None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:32.625040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8315d-1c49-4f1d-bd68-2264d4cdb4ec None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:32.658256 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462819da-6224-44f3-97d0-3ade948caa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:32.658912 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462819da-6224-44f3-97d0-3ade948caa45 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:32.659191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462819da-6224-44f3-97d0-3ade948caa45 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:32.659453 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462819da-6224-44f3-97d0-3ade948caa45 None admin] PATH_INFO: `/v3/users/0ae2743864be426da68c42cb43ca4e1f` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:32.660546 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462819da-6224-44f3-97d0-3ade948caa45 None admin] RBAC: Authorizing `identity:get_user(user_id=0ae2743864be426da68c42cb43ca4e1f)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:32.661968 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462819da-6224-44f3-97d0-3ade948caa45 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:32.663568 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 553/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:32 2022] GET /identity/v3/users/0ae2743864be426da68c42cb43ca4e1f => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:32.670088 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d388f19-a871-47d5-8eb7-dfaff8c5b591 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:32.720955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:32.765833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d388f19-a871-47d5-8eb7-dfaff8c5b591 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:32.798605 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01020d-e626-43b2-b390-c444a53b9198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:32.799230 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01020d-e626-43b2-b390-c444a53b9198 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:32.799507 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01020d-e626-43b2-b390-c444a53b9198 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:32.799776 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01020d-e626-43b2-b390-c444a53b9198 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:32.864478 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01020d-e626-43b2-b390-c444a53b9198 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:32.865914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e01020d-e626-43b2-b390-c444a53b9198 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:32.915525 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:32.961716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 575/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:46:32.967688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462819da-6224-44f3-97d0-3ade948caa45 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:33.024459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:33.068028 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462819da-6224-44f3-97d0-3ade948caa45 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:33.102135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f361e-629d-4c2e-a50a-9e656386f367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:33.102789 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f361e-629d-4c2e-a50a-9e656386f367 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:33.103077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f361e-629d-4c2e-a50a-9e656386f367 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:33.103344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f361e-629d-4c2e-a50a-9e656386f367 None admin] PATH_INFO: `/v3/projects/5c2eddde651e4e15a5fc9a302408a457` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:33.104474 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f361e-629d-4c2e-a50a-9e656386f367 None admin] RBAC: Authorizing `identity:get_project(project_id=5c2eddde651e4e15a5fc9a302408a457)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:33.105893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f361e-629d-4c2e-a50a-9e656386f367 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:33.107580 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 554/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:32 2022] GET /identity/v3/projects/5c2eddde651e4e15a5fc9a302408a457 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:33.113902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01020d-e626-43b2-b390-c444a53b9198 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:33.210881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01020d-e626-43b2-b390-c444a53b9198 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:33.243598 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d6292-205a-4ef8-b308-2be875d34b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:33.244186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d6292-205a-4ef8-b308-2be875d34b99 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:33.244460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d6292-205a-4ef8-b308-2be875d34b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:33.244723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d6292-205a-4ef8-b308-2be875d34b99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:33.245425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d6292-205a-4ef8-b308-2be875d34b99 None admin] RBAC: Adding query filter params (role.id=1b632105fdba460b809a2ecd1d186409, scope.project.id=5c2eddde651e4e15a5fc9a302408a457, user.id=0ae2743864be426da68c42cb43ca4e1f) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:33.245726 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d6292-205a-4ef8-b308-2be875d34b99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:33.247167 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d6292-205a-4ef8-b308-2be875d34b99 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:33.253197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 576/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:46:33 2022] GET /identity/v3/role_assignments?user.id=0ae2743864be426da68c42cb43ca4e1f&scope.project.id=5c2eddde651e4e15a5fc9a302408a457&role.id=1b632105fdba460b809a2ecd1d186409 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:35.438043 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333db5b1-7670-4897-b37c-31e7cb6b1887 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:35.438562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333db5b1-7670-4897-b37c-31e7cb6b1887 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:35.438562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333db5b1-7670-4897-b37c-31e7cb6b1887 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:35.439945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 555/1131] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:35.445179 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d15c88-b986-4acc-aa55-d6d67c8f4c9e None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:35.445594 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d15c88-b986-4acc-aa55-d6d67c8f4c9e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:35.445727 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d15c88-b986-4acc-aa55-d6d67c8f4c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:35.466134 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-25d15c88-b986-4acc-aa55-d6d67c8f4c9e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:35.467860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-25d15c88-b986-4acc-aa55-d6d67c8f4c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:35.478772 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:35.533893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 577/1132] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:46:35.810773 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333db5b1-7670-4897-b37c-31e7cb6b1887 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:35.918921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333db5b1-7670-4897-b37c-31e7cb6b1887 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:35.943218 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:35.954539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac5744-ea58-460d-ba6e-4af0ee8d5234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:35.955242 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac5744-ea58-460d-ba6e-4af0ee8d5234 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:35.955559 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac5744-ea58-460d-ba6e-4af0ee8d5234 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:35.955832 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac5744-ea58-460d-ba6e-4af0ee8d5234 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:35.956949 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac5744-ea58-460d-ba6e-4af0ee8d5234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:35.958383 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac5744-ea58-460d-ba6e-4af0ee8d5234 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:35.960003 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 556/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:35.965972 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d15c88-b986-4acc-aa55-d6d67c8f4c9e None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:36.063514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d15c88-b986-4acc-aa55-d6d67c8f4c9e None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:36.096244 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e06021-d60d-4627-994b-f3ae1c2dea51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:36.097117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e06021-d60d-4627-994b-f3ae1c2dea51 None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:36.097393 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e06021-d60d-4627-994b-f3ae1c2dea51 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:36.097663 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e06021-d60d-4627-994b-f3ae1c2dea51 None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:36.098329 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e06021-d60d-4627-994b-f3ae1c2dea51 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:36.099785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e06021-d60d-4627-994b-f3ae1c2dea51 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:36.121546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:36.129801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 578/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Sep 22 14:46:35 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:46:38.282639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd87ec-4b9b-4456-8eba-e5720f156ca9 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:38.283188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd87ec-4b9b-4456-8eba-e5720f156ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:38.283309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd87ec-4b9b-4456-8eba-e5720f156ca9 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:38.284546 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 557/1135] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:38.289896 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f588e2-dc22-463f-b863-de33951198e7 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:38.290289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f588e2-dc22-463f-b863-de33951198e7 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:38.290446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f588e2-dc22-463f-b863-de33951198e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:38.309199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-86f588e2-dc22-463f-b863-de33951198e7 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:38.310405 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-86f588e2-dc22-463f-b863-de33951198e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:38.376567 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 579/1136] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:38.646360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd87ec-4b9b-4456-8eba-e5720f156ca9 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:38.745590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd87ec-4b9b-4456-8eba-e5720f156ca9 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:38.781448 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dea45c-2f70-4f0a-a67c-5e5b5b637f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:38.782072 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dea45c-2f70-4f0a-a67c-5e5b5b637f33 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:38.782352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dea45c-2f70-4f0a-a67c-5e5b5b637f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:38.782624 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dea45c-2f70-4f0a-a67c-5e5b5b637f33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:38.783773 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dea45c-2f70-4f0a-a67c-5e5b5b637f33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:38.785206 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dea45c-2f70-4f0a-a67c-5e5b5b637f33 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:38.786794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 558/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:46:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:38.792777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f588e2-dc22-463f-b863-de33951198e7 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:38.890921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f588e2-dc22-463f-b863-de33951198e7 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:38.917356 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:38.925435 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84f817-c597-4def-9ef1-2f0b760b2676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:38.926041 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84f817-c597-4def-9ef1-2f0b760b2676 None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:38.926400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84f817-c597-4def-9ef1-2f0b760b2676 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:38.926592 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84f817-c597-4def-9ef1-2f0b760b2676 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:38.927277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84f817-c597-4def-9ef1-2f0b760b2676 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:38.928692 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84f817-c597-4def-9ef1-2f0b760b2676 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:38.949403 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 580/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:46:38 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:46:41.131493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2efce-32f8-4610-b6cf-f93de673a983 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:41.132090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2efce-32f8-4610-b6cf-f93de673a983 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:41.132090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d2efce-32f8-4610-b6cf-f93de673a983 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:41.133361 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 559/1139] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:41.138683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b389d-2881-437d-8bd9-2b68cd167249 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:41.139146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b389d-2881-437d-8bd9-2b68cd167249 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:41.139252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b389d-2881-437d-8bd9-2b68cd167249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:41.157035 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-4d2b389d-2881-437d-8bd9-2b68cd167249 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:41.158225 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-4d2b389d-2881-437d-8bd9-2b68cd167249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:41.224579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 581/1140] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:41.493749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2efce-32f8-4610-b6cf-f93de673a983 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:41.557243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:41.599982 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d2efce-32f8-4610-b6cf-f93de673a983 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:41.632851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09707390-6b68-4069-b936-d656594880ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:41.633476 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09707390-6b68-4069-b936-d656594880ec None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:41.633755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09707390-6b68-4069-b936-d656594880ec None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:41.634022 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09707390-6b68-4069-b936-d656594880ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:41.644412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09707390-6b68-4069-b936-d656594880ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:41.649196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-09707390-6b68-4069-b936-d656594880ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:41.649196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:41.649196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:41.649196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:41.649196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:41.649196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:41.649196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:41.649196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:41.649196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:41.649196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:41.649196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:41.649196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:41.649196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:41.649196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:41.649196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:41.649196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:41.650401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:41.650401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:41.650401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:41.650401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:41.650401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:41.650401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:41.650401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:41.650401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:41.650401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:41.650401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:41.650401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:41.650401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:41.650401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:41.650401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:41.650401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:41.650401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:41.651466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:41.651466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:41.651466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:41.651466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:41.651466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:41.651466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:41.651466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:41.651466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:41.651466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:41.651466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:41.651466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:41.651466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:41.651466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 560/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:46:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:41.656980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b389d-2881-437d-8bd9-2b68cd167249 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:41.752976 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b389d-2881-437d-8bd9-2b68cd167249 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:41.768045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:41.788607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a7b72-162b-4bd2-91b8-198cb9bf52e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:41.789240 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a7b72-162b-4bd2-91b8-198cb9bf52e0 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:41.789510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a7b72-162b-4bd2-91b8-198cb9bf52e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:41.789776 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a7b72-162b-4bd2-91b8-198cb9bf52e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:41.790420 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a7b72-162b-4bd2-91b8-198cb9bf52e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:41.790719 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a7b72-162b-4bd2-91b8-198cb9bf52e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:41.792093 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a7b72-162b-4bd2-91b8-198cb9bf52e0 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:41.800006 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 582/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:46:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:41.806004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09707390-6b68-4069-b936-d656594880ec None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:41.909644 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09707390-6b68-4069-b936-d656594880ec None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:41.929108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:41.943647 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73458422-7523-43ea-9f2f-6a1a88ad572c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:41.944273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73458422-7523-43ea-9f2f-6a1a88ad572c None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:41.944550 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73458422-7523-43ea-9f2f-6a1a88ad572c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:41.944819 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73458422-7523-43ea-9f2f-6a1a88ad572c None admin] PATH_INFO: `/v3/users/e91d9719cf5342b0b5956c33d7c432db` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:41.958629 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73458422-7523-43ea-9f2f-6a1a88ad572c None admin] RBAC: Authorizing `identity:get_user(user_id=e91d9719cf5342b0b5956c33d7c432db)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:41.960100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73458422-7523-43ea-9f2f-6a1a88ad572c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:41.961706 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 561/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:41 2022] GET /identity/v3/users/e91d9719cf5342b0b5956c33d7c432db => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:41.967610 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a7b72-162b-4bd2-91b8-198cb9bf52e0 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:42.064399 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a7b72-162b-4bd2-91b8-198cb9bf52e0 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:42.097072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062ac71-bf1f-4157-b7a2-4ac80b76eab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:42.097666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062ac71-bf1f-4157-b7a2-4ac80b76eab0 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:42.097946 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062ac71-bf1f-4157-b7a2-4ac80b76eab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:42.098219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062ac71-bf1f-4157-b7a2-4ac80b76eab0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:42.142098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:42.164612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062ac71-bf1f-4157-b7a2-4ac80b76eab0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:42.166097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062ac71-bf1f-4157-b7a2-4ac80b76eab0 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:42.268067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 583/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:46:42.274025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73458422-7523-43ea-9f2f-6a1a88ad572c None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:42.371111 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73458422-7523-43ea-9f2f-6a1a88ad572c None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:42.403889 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac2faa-abc8-4189-9484-408c052bf156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:42.404553 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac2faa-abc8-4189-9484-408c052bf156 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:42.404830 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac2faa-abc8-4189-9484-408c052bf156 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:42.405102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac2faa-abc8-4189-9484-408c052bf156 None admin] PATH_INFO: `/v3/projects/d817d764f87d42979d61cb666dd6ec92` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:42.406247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac2faa-abc8-4189-9484-408c052bf156 None admin] RBAC: Authorizing `identity:get_project(project_id=d817d764f87d42979d61cb666dd6ec92)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:42.407714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac2faa-abc8-4189-9484-408c052bf156 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:42.409378 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 562/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:42 2022] GET /identity/v3/projects/d817d764f87d42979d61cb666dd6ec92 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:42.415549 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062ac71-bf1f-4157-b7a2-4ac80b76eab0 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:42.491150 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:42.514103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062ac71-bf1f-4157-b7a2-4ac80b76eab0 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:42.548713 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785d5fc-ab8f-40a5-a87d-46eedb4decf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:42.549331 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785d5fc-ab8f-40a5-a87d-46eedb4decf7 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:42.549600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785d5fc-ab8f-40a5-a87d-46eedb4decf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:42.549905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785d5fc-ab8f-40a5-a87d-46eedb4decf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:42.550644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785d5fc-ab8f-40a5-a87d-46eedb4decf7 None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=d817d764f87d42979d61cb666dd6ec92, user.id=e91d9719cf5342b0b5956c33d7c432db) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:42.550983 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785d5fc-ab8f-40a5-a87d-46eedb4decf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:42.552419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785d5fc-ab8f-40a5-a87d-46eedb4decf7 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:42.560833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 584/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:46:42 2022] GET /identity/v3/role_assignments?user.id=e91d9719cf5342b0b5956c33d7c432db&scope.project.id=d817d764f87d42979d61cb666dd6ec92&role.id=32c8a81e339a462e973e76bb528ef35a => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:44.688309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781340d-e88f-4985-9528-92c747a31bcb None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:44.688821 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781340d-e88f-4985-9528-92c747a31bcb None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:44.688821 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781340d-e88f-4985-9528-92c747a31bcb None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:44.690088 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 563/1147] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:44.695388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d38280-0be5-4bb0-95d3-5d8d7ff4dbcc None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:44.695810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d38280-0be5-4bb0-95d3-5d8d7ff4dbcc None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:44.695936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d38280-0be5-4bb0-95d3-5d8d7ff4dbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:44.714527 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-32d38280-0be5-4bb0-95d3-5d8d7ff4dbcc None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:44.715809 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-32d38280-0be5-4bb0-95d3-5d8d7ff4dbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:44.782633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 585/1148] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:45.053452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781340d-e88f-4985-9528-92c747a31bcb None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:45.101933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:45.156079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781340d-e88f-4985-9528-92c747a31bcb None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:45.189772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61c50e-6915-4c74-82a6-943a63aea6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:45.190398 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61c50e-6915-4c74-82a6-943a63aea6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:45.190676 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61c50e-6915-4c74-82a6-943a63aea6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:45.190973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61c50e-6915-4c74-82a6-943a63aea6b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:45.201504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61c50e-6915-4c74-82a6-943a63aea6b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:45.206287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-eb61c50e-6915-4c74-82a6-943a63aea6b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:45.206287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:45.206287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:45.206287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:45.206287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:45.206287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:45.206287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:45.206287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:45.206287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:45.206287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:45.206287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:45.206287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:45.206287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:45.206287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:45.206287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:45.206287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:45.207454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:45.207454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:45.207454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:45.207454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:45.207454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:45.207454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:45.207454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:45.207454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:45.207454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:45.207454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:45.207454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:45.207454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:45.207454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:45.207454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:45.207454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:45.207454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:45.208456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:45.208456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:45.208456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:45.208456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:45.208456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:45.208456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:45.208456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:45.208456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:45.208456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:45.208456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:45.208456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:45.208456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:45.208456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 564/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:46:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:45.213581 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d38280-0be5-4bb0-95d3-5d8d7ff4dbcc None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:45.279343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:45.310606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d38280-0be5-4bb0-95d3-5d8d7ff4dbcc None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:45.344553 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd46f1-2a27-43cb-939a-f06f73994ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:45.345156 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd46f1-2a27-43cb-939a-f06f73994ffa None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:45.345430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd46f1-2a27-43cb-939a-f06f73994ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:45.345695 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd46f1-2a27-43cb-939a-f06f73994ffa None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:45.346346 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd46f1-2a27-43cb-939a-f06f73994ffa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:45.346927 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd46f1-2a27-43cb-939a-f06f73994ffa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:45.348272 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd46f1-2a27-43cb-939a-f06f73994ffa None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:45.356039 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 586/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:46:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:45.362291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61c50e-6915-4c74-82a6-943a63aea6b0 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:45.460990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61c50e-6915-4c74-82a6-943a63aea6b0 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:45.467855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:45.496006 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692776d-5979-4957-8290-13288f727192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:45.496639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692776d-5979-4957-8290-13288f727192 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:45.496916 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692776d-5979-4957-8290-13288f727192 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:45.497736 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692776d-5979-4957-8290-13288f727192 None admin] PATH_INFO: `/v3/users/e91d9719cf5342b0b5956c33d7c432db` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:45.498899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692776d-5979-4957-8290-13288f727192 None admin] RBAC: Authorizing `identity:get_user(user_id=e91d9719cf5342b0b5956c33d7c432db)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:45.500335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692776d-5979-4957-8290-13288f727192 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:45.501916 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 565/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:45 2022] GET /identity/v3/users/e91d9719cf5342b0b5956c33d7c432db => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:45.507779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd46f1-2a27-43cb-939a-f06f73994ffa None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:45.604054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd46f1-2a27-43cb-939a-f06f73994ffa None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:45.632565 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:45.637882 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199552c3-aaff-484c-8c3e-6f489e878bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:45.638502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199552c3-aaff-484c-8c3e-6f489e878bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:45.638807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199552c3-aaff-484c-8c3e-6f489e878bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:45.639081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199552c3-aaff-484c-8c3e-6f489e878bd8 None admin] PATH_INFO: `/v3/projects/d817d764f87d42979d61cb666dd6ec92` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:45.640189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199552c3-aaff-484c-8c3e-6f489e878bd8 None admin] RBAC: Authorizing `identity:get_project(project_id=d817d764f87d42979d61cb666dd6ec92)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:45.641603 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199552c3-aaff-484c-8c3e-6f489e878bd8 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:45.643258 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 587/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:45 2022] GET /identity/v3/projects/d817d764f87d42979d61cb666dd6ec92 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:45.649226 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692776d-5979-4957-8290-13288f727192 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:45.747688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692776d-5979-4957-8290-13288f727192 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:45.780381 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efe973-a947-4176-b738-27096cfceb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:45.781023 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efe973-a947-4176-b738-27096cfceb70 None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:45.781297 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efe973-a947-4176-b738-27096cfceb70 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:45.781565 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efe973-a947-4176-b738-27096cfceb70 None admin] PATH_INFO: `/v3/projects/d817d764f87d42979d61cb666dd6ec92/users/e91d9719cf5342b0b5956c33d7c432db/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:45.783459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efe973-a947-4176-b738-27096cfceb70 None admin] RBAC: Authorizing `identity:create_grant(project_id=d817d764f87d42979d61cb666dd6ec92, user_id=e91d9719cf5342b0b5956c33d7c432db, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:45.784884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efe973-a947-4176-b738-27096cfceb70 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:45.798378 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 566/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:46:45 2022] PUT /identity/v3/projects/d817d764f87d42979d61cb666dd6ec92/users/e91d9719cf5342b0b5956c33d7c432db/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:46:48.008464 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc43743-f763-4a25-99d4-34079d3fdb13 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:48.009030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc43743-f763-4a25-99d4-34079d3fdb13 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:48.009030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc43743-f763-4a25-99d4-34079d3fdb13 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:48.010348 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 588/1154] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:48.016362 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615b1b3-ad5a-418a-86e0-d07e1062f04f None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:48.016802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615b1b3-ad5a-418a-86e0-d07e1062f04f None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:48.016913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615b1b3-ad5a-418a-86e0-d07e1062f04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:48.034794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-9615b1b3-ad5a-418a-86e0-d07e1062f04f None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:48.036019 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-9615b1b3-ad5a-418a-86e0-d07e1062f04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:48.064250 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:48.106211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 567/1155] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:48.383647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc43743-f763-4a25-99d4-34079d3fdb13 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:48.485214 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc43743-f763-4a25-99d4-34079d3fdb13 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:48.519015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b81a6-096a-45d8-bb92-7f057da274c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:48.519633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b81a6-096a-45d8-bb92-7f057da274c8 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:48.519911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b81a6-096a-45d8-bb92-7f057da274c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:48.520189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b81a6-096a-45d8-bb92-7f057da274c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:48.531135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b81a6-096a-45d8-bb92-7f057da274c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:46:48.535943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-272b81a6-096a-45d8-bb92-7f057da274c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:48.535943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:48.535943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:48.535943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:48.535943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:48.535943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:48.535943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:48.535943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:48.535943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:48.535943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:48.535943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:48.535943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:48.535943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:46:48.535943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:46:48.535943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:46:48.535943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:46:48.537036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:46:48.537036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:46:48.537036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:46:48.537036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:46:48.537036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:48.537036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:48.537036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:48.537036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:48.537036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:48.537036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:48.537036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:48.537036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:48.537036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:48.537036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:48.537036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:48.537036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:48.538089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:48.538089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:48.538089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:48.538089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:48.538089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:46:48.538089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:46:48.538089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:46:48.538089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:46:48.538089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:46:48.538089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:46:48.538089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:46:48.538089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:46:48.538089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 589/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:46:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:48.543134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615b1b3-ad5a-418a-86e0-d07e1062f04f None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:48.649758 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615b1b3-ad5a-418a-86e0-d07e1062f04f None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:48.685276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a168d-f7f7-40b5-a69d-8ae6b3354143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:48.685885 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a168d-f7f7-40b5-a69d-8ae6b3354143 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:48.686159 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a168d-f7f7-40b5-a69d-8ae6b3354143 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:48.686424 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a168d-f7f7-40b5-a69d-8ae6b3354143 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:48.687114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a168d-f7f7-40b5-a69d-8ae6b3354143 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:48.687413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a168d-f7f7-40b5-a69d-8ae6b3354143 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:48.688849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a168d-f7f7-40b5-a69d-8ae6b3354143 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:48.697378 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 568/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:46:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:48.703906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b81a6-096a-45d8-bb92-7f057da274c8 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:48.761842 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:48.802370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b81a6-096a-45d8-bb92-7f057da274c8 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:48.835853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617e9d5-acb4-4418-a8bc-ad7e088b278d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:48.836482 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617e9d5-acb4-4418-a8bc-ad7e088b278d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:48.836761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617e9d5-acb4-4418-a8bc-ad7e088b278d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:48.837033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617e9d5-acb4-4418-a8bc-ad7e088b278d None admin] PATH_INFO: `/v3/users/e91d9719cf5342b0b5956c33d7c432db` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:48.838152 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617e9d5-acb4-4418-a8bc-ad7e088b278d None admin] RBAC: Authorizing `identity:get_user(user_id=e91d9719cf5342b0b5956c33d7c432db)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:48.839608 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617e9d5-acb4-4418-a8bc-ad7e088b278d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:48.841245 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 590/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:46:48 2022] GET /identity/v3/users/e91d9719cf5342b0b5956c33d7c432db => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:48.847626 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a168d-f7f7-40b5-a69d-8ae6b3354143 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:48.875789 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:48.962039 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a168d-f7f7-40b5-a69d-8ae6b3354143 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:49.007674 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a5bd1-ac2f-4176-b47d-e92a585b2868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:49.008555 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a5bd1-ac2f-4176-b47d-e92a585b2868 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:49.008964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a5bd1-ac2f-4176-b47d-e92a585b2868 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:49.009361 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a5bd1-ac2f-4176-b47d-e92a585b2868 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:49.082394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a5bd1-ac2f-4176-b47d-e92a585b2868 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:49.083886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a5bd1-ac2f-4176-b47d-e92a585b2868 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:49.110259 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:49.181388 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 569/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:46:49.187955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617e9d5-acb4-4418-a8bc-ad7e088b278d None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:49.292212 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617e9d5-acb4-4418-a8bc-ad7e088b278d None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:49.318476 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:49.331219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b20a0-23ad-42c4-9876-fbadcd77726c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:49.331963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b20a0-23ad-42c4-9876-fbadcd77726c None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:49.332237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b20a0-23ad-42c4-9876-fbadcd77726c None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:49.332500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b20a0-23ad-42c4-9876-fbadcd77726c None admin] PATH_INFO: `/v3/projects/d817d764f87d42979d61cb666dd6ec92` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:49.333622 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b20a0-23ad-42c4-9876-fbadcd77726c None admin] RBAC: Authorizing `identity:get_project(project_id=d817d764f87d42979d61cb666dd6ec92)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:49.335091 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b20a0-23ad-42c4-9876-fbadcd77726c None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:49.336727 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 591/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:46:49 2022] GET /identity/v3/projects/d817d764f87d42979d61cb666dd6ec92 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:49.343497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a5bd1-ac2f-4176-b47d-e92a585b2868 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:49.441910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a5bd1-ac2f-4176-b47d-e92a585b2868 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:49.468198 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:49.477339 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8ea56-2171-4577-9d37-4ba42feeb94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:49.477944 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8ea56-2171-4577-9d37-4ba42feeb94e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:49.478266 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8ea56-2171-4577-9d37-4ba42feeb94e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:49.478495 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8ea56-2171-4577-9d37-4ba42feeb94e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:49.479241 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8ea56-2171-4577-9d37-4ba42feeb94e None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=d817d764f87d42979d61cb666dd6ec92, user.id=e91d9719cf5342b0b5956c33d7c432db) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:49.479547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8ea56-2171-4577-9d37-4ba42feeb94e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:49.480970 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de8ea56-2171-4577-9d37-4ba42feeb94e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:49.487033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 570/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:46:49 2022] GET /identity/v3/role_assignments?user.id=e91d9719cf5342b0b5956c33d7c432db&scope.project.id=d817d764f87d42979d61cb666dd6ec92&role.id=32c8a81e339a462e973e76bb528ef35a => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:51.667002 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0be3e4-45b3-458b-8721-a2d115c89361 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:51.667514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0be3e4-45b3-458b-8721-a2d115c89361 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:51.667514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0be3e4-45b3-458b-8721-a2d115c89361 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:51.668841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 592/1162] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:51.674242 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1056a54-8d66-4edd-bea5-c4a8f42dbbf3 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:51.674630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1056a54-8d66-4edd-bea5-c4a8f42dbbf3 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:51.674799 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1056a54-8d66-4edd-bea5-c4a8f42dbbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:51.692915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-f1056a54-8d66-4edd-bea5-c4a8f42dbbf3 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:51.694086 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f1056a54-8d66-4edd-bea5-c4a8f42dbbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:51.758127 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 571/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:52.027134 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0be3e4-45b3-458b-8721-a2d115c89361 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:52.126721 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0be3e4-45b3-458b-8721-a2d115c89361 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:52.160556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ab4e9-3fa3-4ed6-9aa5-e9545cc4856f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:52.161242 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ab4e9-3fa3-4ed6-9aa5-e9545cc4856f None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:52.161574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ab4e9-3fa3-4ed6-9aa5-e9545cc4856f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:52.161853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ab4e9-3fa3-4ed6-9aa5-e9545cc4856f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:52.222911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:52.229909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ab4e9-3fa3-4ed6-9aa5-e9545cc4856f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:52.231925 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ab4e9-3fa3-4ed6-9aa5-e9545cc4856f None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:52.328128 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 593/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:46:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 14:46:52.334151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1056a54-8d66-4edd-bea5-c4a8f42dbbf3 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:52.429456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1056a54-8d66-4edd-bea5-c4a8f42dbbf3 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:52.449573 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:52.466211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08853e59-d97d-48de-8468-0afc0d330dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:52.466886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08853e59-d97d-48de-8468-0afc0d330dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:52.467167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08853e59-d97d-48de-8468-0afc0d330dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:52.467438 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08853e59-d97d-48de-8468-0afc0d330dc9 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:52.474382 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08853e59-d97d-48de-8468-0afc0d330dc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:52.475854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08853e59-d97d-48de-8468-0afc0d330dc9 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:52.487777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-08853e59-d97d-48de-8468-0afc0d330dc9 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 22 14:46:52.487777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:52.487777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 22 14:46:52.487777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 22 14:46:52.487777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:46:52.487777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:46:52.487777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:46:52.487777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:46:52.487777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 22 14:46:52.487777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:52.487777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:46:52.487777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:52.487777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:46:52.487777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:46:52.487777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:46:52.487777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:46:52.487777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:46:52.489655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:46:52.489655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:46:52.489655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:46:52.489655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:46:52.489655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:46:52.489655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:46:52.489655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:46:52.489655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:46:52.489655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:46:52.489655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:46:52.489655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:46:52.489655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:46:52.489655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:46:52.489655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:46:52.489655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:46:52.489655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:46:52.491294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:46:52.491294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:46:52.491294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:46:52.491294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:46:52.491294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:46:52.491294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:46:52.491294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:46:52.491294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:46:52.491294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:46:52.491294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:46:52.491294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 22 14:46:52.491294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:46:52.491294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 22 14:46:52.491294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:46:52.491294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 572/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Sep 22 14:46:52 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:46:52.495056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ab4e9-3fa3-4ed6-9aa5-e9545cc4856f None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:52.585441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:52.592140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ab4e9-3fa3-4ed6-9aa5-e9545cc4856f None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:52.625104 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db031b-8ac4-4f76-b9cd-d1028e9f98c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:52.625693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db031b-8ac4-4f76-b9cd-d1028e9f98c8 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:52.625964 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db031b-8ac4-4f76-b9cd-d1028e9f98c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:52.626230 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db031b-8ac4-4f76-b9cd-d1028e9f98c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:52.626882 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db031b-8ac4-4f76-b9cd-d1028e9f98c8 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:46:52.627180 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db031b-8ac4-4f76-b9cd-d1028e9f98c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:52.628516 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db031b-8ac4-4f76-b9cd-d1028e9f98c8 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:52.635443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 594/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Sep 22 14:46:52 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:54.769437 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776162d-0227-44be-9c79-efd8cb20c256 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:54.769935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776162d-0227-44be-9c79-efd8cb20c256 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:54.769935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776162d-0227-44be-9c79-efd8cb20c256 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:54.771280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 573/1167] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:54.776637 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353ed0a-47c1-4c69-9ca6-6c581e4eaccf None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:54.777067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353ed0a-47c1-4c69-9ca6-6c581e4eaccf None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:54.777189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353ed0a-47c1-4c69-9ca6-6c581e4eaccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:54.794747 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-c353ed0a-47c1-4c69-9ca6-6c581e4eaccf None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:54.795944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-c353ed0a-47c1-4c69-9ca6-6c581e4eaccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:54.860209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 595/1168] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:55.127884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2776162d-0227-44be-9c79-efd8cb20c256 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:55.226397 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2776162d-0227-44be-9c79-efd8cb20c256 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:55.259744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f80880-1553-4a0a-8d14-c7f8fe6cac54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:55.260342 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f80880-1553-4a0a-8d14-c7f8fe6cac54 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:55.260621 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f80880-1553-4a0a-8d14-c7f8fe6cac54 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:55.260894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f80880-1553-4a0a-8d14-c7f8fe6cac54 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:55.261558 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f80880-1553-4a0a-8d14-c7f8fe6cac54 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:55.262981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f80880-1553-4a0a-8d14-c7f8fe6cac54 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:55.278313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 574/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Thu Sep 22 14:46:55 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:46:57.462134 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e298c-6e76-4bbd-845e-7c794e267a0c None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:57.462691 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e298c-6e76-4bbd-845e-7c794e267a0c None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:57.462691 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e298c-6e76-4bbd-845e-7c794e267a0c None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:57.464047 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 596/1170] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:46:57.470003 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84531365-558c-464e-a423-e285dfc923b0 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:57.470445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84531365-558c-464e-a423-e285dfc923b0 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:57.470582 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84531365-558c-464e-a423-e285dfc923b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:57.488696 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-84531365-558c-464e-a423-e285dfc923b0 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:46:57.489911 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-84531365-558c-464e-a423-e285dfc923b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:46:57.523282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:57.559050 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 575/1171] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:46:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:46:57.838652 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e298c-6e76-4bbd-845e-7c794e267a0c None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:57.927722 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:46:57.938935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e298c-6e76-4bbd-845e-7c794e267a0c None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:57.972763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fb64e-d798-421c-9253-d93c2b8d7673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:57.973386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fb64e-d798-421c-9253-d93c2b8d7673 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:57.973663 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fb64e-d798-421c-9253-d93c2b8d7673 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:57.973934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fb64e-d798-421c-9253-d93c2b8d7673 None admin] PATH_INFO: `/v3/domains/e0b40594368f47f9adbbf3c2acf917fe` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:57.984100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fb64e-d798-421c-9253-d93c2b8d7673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e0b40594368f47f9adbbf3c2acf917fe)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:57.985534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fb64e-d798-421c-9253-d93c2b8d7673 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:57.987164 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 597/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Thu Sep 22 14:46:57 2022] GET /identity/v3/domains/e0b40594368f47f9adbbf3c2acf917fe => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:46:57.993040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84531365-558c-464e-a423-e285dfc923b0 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:46:58.102710 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84531365-558c-464e-a423-e285dfc923b0 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:46:58.136977 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd59cabe-cb23-4c54-8976-630fd4c5118d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:46:58.137765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59cabe-cb23-4c54-8976-630fd4c5118d None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:46:58.138048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59cabe-cb23-4c54-8976-630fd4c5118d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:46:58.138318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59cabe-cb23-4c54-8976-630fd4c5118d None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:46:58.139003 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd59cabe-cb23-4c54-8976-630fd4c5118d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:46:58.140449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd59cabe-cb23-4c54-8976-630fd4c5118d None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:46:58.166231 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 576/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Sep 22 14:46:57 2022] POST /identity/v3/projects => generated 361 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:47:00.366517 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be6218-8a9c-4230-8057-b38180c9100c None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:00.367072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be6218-8a9c-4230-8057-b38180c9100c None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:00.367072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be6218-8a9c-4230-8057-b38180c9100c None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:00.368431 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 598/1174] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:47:00.374825 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888666c6-27ea-4e22-aa9f-c258fd8eee20 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:00.375345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888666c6-27ea-4e22-aa9f-c258fd8eee20 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:00.375446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888666c6-27ea-4e22-aa9f-c258fd8eee20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:00.393686 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-888666c6-27ea-4e22-aa9f-c258fd8eee20 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:47:00.394911 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-888666c6-27ea-4e22-aa9f-c258fd8eee20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:47:00.410997 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:00.461924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 577/1175] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:47:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:47:00.738222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be6218-8a9c-4230-8057-b38180c9100c None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:00.836404 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be6218-8a9c-4230-8057-b38180c9100c None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:00.870096 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183084ba-d7a2-4784-8d62-f0c460fa2111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:00.870719 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183084ba-d7a2-4784-8d62-f0c460fa2111 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:00.871026 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183084ba-d7a2-4784-8d62-f0c460fa2111 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:00.871323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183084ba-d7a2-4784-8d62-f0c460fa2111 None admin] PATH_INFO: `/v3/domains/e0b40594368f47f9adbbf3c2acf917fe` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:00.872441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183084ba-d7a2-4784-8d62-f0c460fa2111 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e0b40594368f47f9adbbf3c2acf917fe)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:00.873895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183084ba-d7a2-4784-8d62-f0c460fa2111 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:00.875530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 599/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Thu Sep 22 14:47:00 2022] GET /identity/v3/domains/e0b40594368f47f9adbbf3c2acf917fe => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:00.881507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888666c6-27ea-4e22-aa9f-c258fd8eee20 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:00.982645 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888666c6-27ea-4e22-aa9f-c258fd8eee20 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:01.018397 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd57324-3d40-4a37-a80e-d6cd41fced5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:01.018949 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd57324-3d40-4a37-a80e-d6cd41fced5b None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:01.019229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd57324-3d40-4a37-a80e-d6cd41fced5b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:01.019502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd57324-3d40-4a37-a80e-d6cd41fced5b None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:01.020160 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd57324-3d40-4a37-a80e-d6cd41fced5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:01.021600 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd57324-3d40-4a37-a80e-d6cd41fced5b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:01.045431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 578/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:47:00 2022] POST /identity/v3/users => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:47:03.320280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4ad87-2d71-4268-b133-abd4509eb1d8 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:03.320807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4ad87-2d71-4268-b133-abd4509eb1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:03.320807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4ad87-2d71-4268-b133-abd4509eb1d8 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:03.322199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 600/1178] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:47:03.327668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a0819-050b-49fd-b137-1a19c38b40a0 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:03.328056 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a0819-050b-49fd-b137-1a19c38b40a0 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:03.328220 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a0819-050b-49fd-b137-1a19c38b40a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:03.346041 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-594a0819-050b-49fd-b137-1a19c38b40a0 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:47:03.347513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-594a0819-050b-49fd-b137-1a19c38b40a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:47:03.349446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:03.417467 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 579/1179] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:47:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:47:03.688914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4ad87-2d71-4268-b133-abd4509eb1d8 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:03.708288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:03.790756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4ad87-2d71-4268-b133-abd4509eb1d8 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:03.824077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070a784-a14e-448c-aff4-39a208548690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:03.824736 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070a784-a14e-448c-aff4-39a208548690 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:03.825018 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070a784-a14e-448c-aff4-39a208548690 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:03.825291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070a784-a14e-448c-aff4-39a208548690 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:03.835674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f070a784-a14e-448c-aff4-39a208548690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:47:03.841015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-f070a784-a14e-448c-aff4-39a208548690 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:47:03.841015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:47:03.841015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:47:03.841015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:47:03.841015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:47:03.841015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:47:03.841015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:47:03.841015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:47:03.841015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:47:03.841015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:47:03.841015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:47:03.841015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:47:03.841015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:47:03.841015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:47:03.841015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:47:03.841015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:47:03.842098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:47:03.842098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:47:03.842098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:47:03.842098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:47:03.842098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:47:03.842098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:47:03.842098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:47:03.842098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:47:03.842098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:47:03.842098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:47:03.842098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:47:03.842098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:47:03.842098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:47:03.842098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:47:03.842098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:47:03.842098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:47:03.843071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:47:03.843071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:47:03.843071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:47:03.843071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:47:03.843071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:47:03.843071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:47:03.843071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:47:03.843071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:47:03.843071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:47:03.843071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:47:03.843071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:47:03.843071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:47:03.843071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 601/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:47:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:47:03.848529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a0819-050b-49fd-b137-1a19c38b40a0 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:03.946071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a0819-050b-49fd-b137-1a19c38b40a0 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:03.981304 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45576412-5274-4a21-b0d3-5d057a582755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:03.981915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45576412-5274-4a21-b0d3-5d057a582755 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:03.982220 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45576412-5274-4a21-b0d3-5d057a582755 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:03.982496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45576412-5274-4a21-b0d3-5d057a582755 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:03.983190 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45576412-5274-4a21-b0d3-5d057a582755 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:47:03.983495 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45576412-5274-4a21-b0d3-5d057a582755 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:03.984862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45576412-5274-4a21-b0d3-5d057a582755 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:03.989828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:03.996835 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 580/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:47:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:03.999473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070a784-a14e-448c-aff4-39a208548690 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:04.074104 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:04.095503 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070a784-a14e-448c-aff4-39a208548690 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:04.127557 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38f62e-a5b9-4d84-8dd4-f9864f10750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:04.128169 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38f62e-a5b9-4d84-8dd4-f9864f10750d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:04.128443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38f62e-a5b9-4d84-8dd4-f9864f10750d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:04.128711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38f62e-a5b9-4d84-8dd4-f9864f10750d None admin] PATH_INFO: `/v3/users/5f6720e38691461ea7137c4d51e001b9` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:04.141692 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38f62e-a5b9-4d84-8dd4-f9864f10750d None admin] RBAC: Authorizing `identity:get_user(user_id=5f6720e38691461ea7137c4d51e001b9)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:04.143316 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38f62e-a5b9-4d84-8dd4-f9864f10750d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:04.144923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 602/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:47:03 2022] GET /identity/v3/users/5f6720e38691461ea7137c4d51e001b9 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:04.150934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45576412-5274-4a21-b0d3-5d057a582755 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:04.250541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45576412-5274-4a21-b0d3-5d057a582755 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:04.284276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753534fb-83a5-4e35-866d-e3a6acf76198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:04.284877 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753534fb-83a5-4e35-866d-e3a6acf76198 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:04.285148 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753534fb-83a5-4e35-866d-e3a6acf76198 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:04.285412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753534fb-83a5-4e35-866d-e3a6acf76198 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:04.349670 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:04.351947 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753534fb-83a5-4e35-866d-e3a6acf76198 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:04.353742 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753534fb-83a5-4e35-866d-e3a6acf76198 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:04.452730 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 581/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:47:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:47:04.458750 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38f62e-a5b9-4d84-8dd4-f9864f10750d None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:04.554923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38f62e-a5b9-4d84-8dd4-f9864f10750d None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:04.588716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579ab53-9a51-4183-8152-b5d0e9bc93f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:04.589335 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579ab53-9a51-4183-8152-b5d0e9bc93f0 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:04.589615 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579ab53-9a51-4183-8152-b5d0e9bc93f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:04.589952 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579ab53-9a51-4183-8152-b5d0e9bc93f0 None admin] PATH_INFO: `/v3/projects/11c134d2ec164bda85dac643bc0bd5d4` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:04.591106 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579ab53-9a51-4183-8152-b5d0e9bc93f0 None admin] RBAC: Authorizing `identity:get_project(project_id=11c134d2ec164bda85dac643bc0bd5d4)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:04.592524 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579ab53-9a51-4183-8152-b5d0e9bc93f0 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:04.594148 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 603/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:47:04 2022] GET /identity/v3/projects/11c134d2ec164bda85dac643bc0bd5d4 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:04.600632 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753534fb-83a5-4e35-866d-e3a6acf76198 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:04.694595 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:04.697838 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753534fb-83a5-4e35-866d-e3a6acf76198 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:04.731431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7e2f2-fc20-45b1-bf1b-f78142771181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:04.732020 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7e2f2-fc20-45b1-bf1b-f78142771181 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:04.732293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7e2f2-fc20-45b1-bf1b-f78142771181 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:04.732557 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7e2f2-fc20-45b1-bf1b-f78142771181 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:04.733263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7e2f2-fc20-45b1-bf1b-f78142771181 None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=11c134d2ec164bda85dac643bc0bd5d4, user.id=5f6720e38691461ea7137c4d51e001b9) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:47:04.733563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7e2f2-fc20-45b1-bf1b-f78142771181 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:04.735001 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7e2f2-fc20-45b1-bf1b-f78142771181 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:04.740138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 582/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:47:04 2022] GET /identity/v3/role_assignments?user.id=5f6720e38691461ea7137c4d51e001b9&scope.project.id=11c134d2ec164bda85dac643bc0bd5d4&role.id=32c8a81e339a462e973e76bb528ef35a => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:06.845801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1dae49-e06b-48cf-8f80-e1adcbecd262 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:06.846621 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1dae49-e06b-48cf-8f80-e1adcbecd262 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:06.846621 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1dae49-e06b-48cf-8f80-e1adcbecd262 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:06.847993 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 604/1186] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:47:06.853061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9def7f0-66a1-4787-b602-ffc87af687d5 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:06.853463 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9def7f0-66a1-4787-b602-ffc87af687d5 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:06.853608 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9def7f0-66a1-4787-b602-ffc87af687d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:06.873035 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-c9def7f0-66a1-4787-b602-ffc87af687d5 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:47:06.874245 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c9def7f0-66a1-4787-b602-ffc87af687d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:47:06.942481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 583/1187] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:47:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:47:07.209504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1dae49-e06b-48cf-8f80-e1adcbecd262 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:07.213311 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:07.306704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1dae49-e06b-48cf-8f80-e1adcbecd262 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:07.340901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7169e58-bebc-407f-9d10-f9b172693dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:07.341523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7169e58-bebc-407f-9d10-f9b172693dda None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:07.341803 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7169e58-bebc-407f-9d10-f9b172693dda None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:07.342078 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7169e58-bebc-407f-9d10-f9b172693dda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:07.352225 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7169e58-bebc-407f-9d10-f9b172693dda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:47:07.358440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-e7169e58-bebc-407f-9d10-f9b172693dda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:47:07.358440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:47:07.358440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:47:07.358440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:47:07.358440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:47:07.358440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:47:07.358440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:47:07.358440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:47:07.358440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:47:07.358440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:47:07.358440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:47:07.358440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:47:07.358440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:47:07.358440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:47:07.358440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:47:07.358440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:47:07.359600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:47:07.359600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:47:07.359600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:47:07.359600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:47:07.359600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:47:07.359600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:47:07.359600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:47:07.359600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:47:07.359600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:47:07.359600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:47:07.359600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:47:07.359600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:47:07.359600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:47:07.359600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:47:07.359600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:47:07.359600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:47:07.361602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:47:07.361602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:47:07.361602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:47:07.361602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:47:07.361602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:47:07.361602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:47:07.361602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:47:07.361602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:47:07.361602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:47:07.361602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:47:07.361602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:47:07.361602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:47:07.361602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 605/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:47:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:47:07.366266 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9def7f0-66a1-4787-b602-ffc87af687d5 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:07.456458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:07.463817 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9def7f0-66a1-4787-b602-ffc87af687d5 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:07.497679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7bddb-6035-461c-84d2-c6cf6ff656ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:07.498277 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7bddb-6035-461c-84d2-c6cf6ff656ec None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:07.498553 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7bddb-6035-461c-84d2-c6cf6ff656ec None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:07.498857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7bddb-6035-461c-84d2-c6cf6ff656ec None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:07.499513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7bddb-6035-461c-84d2-c6cf6ff656ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:47:07.499812 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7bddb-6035-461c-84d2-c6cf6ff656ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:07.501201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7bddb-6035-461c-84d2-c6cf6ff656ec None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:07.509040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 584/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:47:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:07.515131 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7169e58-bebc-407f-9d10-f9b172693dda None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:07.576448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:07.613488 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7169e58-bebc-407f-9d10-f9b172693dda None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:07.645958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e466d-d915-4fe5-ad64-e243322b17f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:07.646583 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e466d-d915-4fe5-ad64-e243322b17f8 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:07.646900 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e466d-d915-4fe5-ad64-e243322b17f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:07.647175 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e466d-d915-4fe5-ad64-e243322b17f8 None admin] PATH_INFO: `/v3/users/5f6720e38691461ea7137c4d51e001b9` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:07.648295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e466d-d915-4fe5-ad64-e243322b17f8 None admin] RBAC: Authorizing `identity:get_user(user_id=5f6720e38691461ea7137c4d51e001b9)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:07.649715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089e466d-d915-4fe5-ad64-e243322b17f8 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:07.651323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 606/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:47:07 2022] GET /identity/v3/users/5f6720e38691461ea7137c4d51e001b9 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:07.657877 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7bddb-6035-461c-84d2-c6cf6ff656ec None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:07.758075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7bddb-6035-461c-84d2-c6cf6ff656ec None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:07.791105 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c642ce-c22b-4ded-a727-fc465f94932d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:07.791727 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c642ce-c22b-4ded-a727-fc465f94932d None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:07.792008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c642ce-c22b-4ded-a727-fc465f94932d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:07.792282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c642ce-c22b-4ded-a727-fc465f94932d None admin] PATH_INFO: `/v3/projects/11c134d2ec164bda85dac643bc0bd5d4` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:07.793993 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c642ce-c22b-4ded-a727-fc465f94932d None admin] RBAC: Authorizing `identity:get_project(project_id=11c134d2ec164bda85dac643bc0bd5d4)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:07.795480 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c642ce-c22b-4ded-a727-fc465f94932d None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:07.797136 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 585/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:47:07 2022] GET /identity/v3/projects/11c134d2ec164bda85dac643bc0bd5d4 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:07.803198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e466d-d915-4fe5-ad64-e243322b17f8 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:07.899882 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e466d-d915-4fe5-ad64-e243322b17f8 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:07.928106 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:07.933185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba0b0b-804e-49b5-b0b9-61c7a8181569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:07.933841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba0b0b-804e-49b5-b0b9-61c7a8181569 None admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:07.934116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba0b0b-804e-49b5-b0b9-61c7a8181569 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:07.934384 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba0b0b-804e-49b5-b0b9-61c7a8181569 None admin] PATH_INFO: `/v3/projects/11c134d2ec164bda85dac643bc0bd5d4/users/5f6720e38691461ea7137c4d51e001b9/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:07.936284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba0b0b-804e-49b5-b0b9-61c7a8181569 None admin] RBAC: Authorizing `identity:create_grant(project_id=11c134d2ec164bda85dac643bc0bd5d4, user_id=5f6720e38691461ea7137c4d51e001b9, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:07.937694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba0b0b-804e-49b5-b0b9-61c7a8181569 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:07.950295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 607/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:47:07 2022] PUT /identity/v3/projects/11c134d2ec164bda85dac643bc0bd5d4/users/5f6720e38691461ea7137c4d51e001b9/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:47:10.046237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86653595-73fc-42d1-b84d-2583f151d524 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:10.046804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86653595-73fc-42d1-b84d-2583f151d524 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:10.046804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86653595-73fc-42d1-b84d-2583f151d524 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:10.048164 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 586/1193] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:47:10.053318 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac20595-896d-4595-8d0d-19d4821b1073 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:10.053739 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac20595-896d-4595-8d0d-19d4821b1073 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:10.053864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac20595-896d-4595-8d0d-19d4821b1073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:10.072172 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-bac20595-896d-4595-8d0d-19d4821b1073 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:47:10.073360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-bac20595-896d-4595-8d0d-19d4821b1073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:47:10.139618 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 608/1194] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:47:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:47:10.406373 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86653595-73fc-42d1-b84d-2583f151d524 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:10.432764 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:10.508787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86653595-73fc-42d1-b84d-2583f151d524 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:10.543581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f13ead-9981-4491-beef-2aa5a846eb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:10.544216 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f13ead-9981-4491-beef-2aa5a846eb74 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:10.544493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f13ead-9981-4491-beef-2aa5a846eb74 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:10.544762 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f13ead-9981-4491-beef-2aa5a846eb74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:10.555397 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f13ead-9981-4491-beef-2aa5a846eb74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:47:10.560158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-81f13ead-9981-4491-beef-2aa5a846eb74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:47:10.560158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:47:10.560158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:47:10.560158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:47:10.560158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:47:10.560158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:47:10.560158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:47:10.560158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:47:10.560158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:47:10.560158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:47:10.560158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:47:10.560158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:47:10.560158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:47:10.560158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:47:10.560158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:47:10.560158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:47:10.561822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:47:10.561822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:47:10.561822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:47:10.561822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:47:10.561822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:47:10.561822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:47:10.561822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:47:10.561822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:47:10.561822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:47:10.561822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:47:10.561822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:47:10.561822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:47:10.561822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:47:10.561822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:47:10.561822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:47:10.561822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:47:10.563354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:47:10.563354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:47:10.563354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:47:10.563354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:47:10.563354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:47:10.563354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:47:10.563354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:47:10.563354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:47:10.563354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:47:10.563354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:47:10.563354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:47:10.563354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:47:10.563354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 587/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:47:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:47:10.567611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac20595-896d-4595-8d0d-19d4821b1073 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:10.662296 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac20595-896d-4595-8d0d-19d4821b1073 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:10.684500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:10.696567 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6b3fa-7036-4e9d-89d1-364b0a9515c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:10.697170 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6b3fa-7036-4e9d-89d1-364b0a9515c1 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:10.697448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6b3fa-7036-4e9d-89d1-364b0a9515c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:10.697722 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6b3fa-7036-4e9d-89d1-364b0a9515c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:10.698377 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6b3fa-7036-4e9d-89d1-364b0a9515c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:47:10.698680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6b3fa-7036-4e9d-89d1-364b0a9515c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:10.700115 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6b3fa-7036-4e9d-89d1-364b0a9515c1 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:10.708024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 609/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:47:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:47:10.714005 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f13ead-9981-4491-beef-2aa5a846eb74 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:10.795969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:10.811381 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f13ead-9981-4491-beef-2aa5a846eb74 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:10.844370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153da21-4ec7-4689-b800-190ae70e9741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:10.844989 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153da21-4ec7-4689-b800-190ae70e9741 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:10.845261 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153da21-4ec7-4689-b800-190ae70e9741 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:10.845528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153da21-4ec7-4689-b800-190ae70e9741 None admin] PATH_INFO: `/v3/users/5f6720e38691461ea7137c4d51e001b9` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:10.846906 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153da21-4ec7-4689-b800-190ae70e9741 None admin] RBAC: Authorizing `identity:get_user(user_id=5f6720e38691461ea7137c4d51e001b9)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:10.848327 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153da21-4ec7-4689-b800-190ae70e9741 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:10.849884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 588/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Sep 22 14:47:10 2022] GET /identity/v3/users/5f6720e38691461ea7137c4d51e001b9 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:10.855738 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6b3fa-7036-4e9d-89d1-364b0a9515c1 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:10.954607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6b3fa-7036-4e9d-89d1-364b0a9515c1 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:10.987262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b085f4-9301-4a3b-bee1-ea71c7f17e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:10.987857 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b085f4-9301-4a3b-bee1-ea71c7f17e1b None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:10.988130 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b085f4-9301-4a3b-bee1-ea71c7f17e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:10.988394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b085f4-9301-4a3b-bee1-ea71c7f17e1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:11.038580 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:11.053567 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b085f4-9301-4a3b-bee1-ea71c7f17e1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:11.055034 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b085f4-9301-4a3b-bee1-ea71c7f17e1b None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:11.151304 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 610/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:47:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:47:11.157864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153da21-4ec7-4689-b800-190ae70e9741 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:11.256140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153da21-4ec7-4689-b800-190ae70e9741 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:11.288574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cbdf5d-a2a3-4ff8-b2d1-1160ae9c782c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:11.289189 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbdf5d-a2a3-4ff8-b2d1-1160ae9c782c None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:11.289467 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbdf5d-a2a3-4ff8-b2d1-1160ae9c782c None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:11.289737 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbdf5d-a2a3-4ff8-b2d1-1160ae9c782c None admin] PATH_INFO: `/v3/projects/11c134d2ec164bda85dac643bc0bd5d4` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:11.290858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cbdf5d-a2a3-4ff8-b2d1-1160ae9c782c None admin] RBAC: Authorizing `identity:get_project(project_id=11c134d2ec164bda85dac643bc0bd5d4)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:11.292280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cbdf5d-a2a3-4ff8-b2d1-1160ae9c782c None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:11.294077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 589/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Sep 22 14:47:11 2022] GET /identity/v3/projects/11c134d2ec164bda85dac643bc0bd5d4 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:47:11.300246 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b085f4-9301-4a3b-bee1-ea71c7f17e1b None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:47:11.382261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:11.398266 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b085f4-9301-4a3b-bee1-ea71c7f17e1b None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:47:11.432522 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5a4d0-a52d-4df2-b121-da544cc0147c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:47:11.433113 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5a4d0-a52d-4df2-b121-da544cc0147c None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:47:11.433446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5a4d0-a52d-4df2-b121-da544cc0147c None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:47:11.433714 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5a4d0-a52d-4df2-b121-da544cc0147c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:47:11.434451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5a4d0-a52d-4df2-b121-da544cc0147c None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=11c134d2ec164bda85dac643bc0bd5d4, user.id=5f6720e38691461ea7137c4d51e001b9) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:47:11.434780 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5a4d0-a52d-4df2-b121-da544cc0147c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:47:11.436191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5a4d0-a52d-4df2-b121-da544cc0147c None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:47:11.441729 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 611/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:47:11 2022] GET /identity/v3/role_assignments?user.id=5f6720e38691461ea7137c4d51e001b9&scope.project.id=11c134d2ec164bda85dac643bc0bd5d4&role.id=32c8a81e339a462e973e76bb528ef35a => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:47:21.284482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=90 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:47:21.439586 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=26 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:07.788348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e8740-2947-4f36-bcee-e432066fc924 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:07.788348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e8740-2947-4f36-bcee-e432066fc924 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:07.788348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e8740-2947-4f36-bcee-e432066fc924 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:07.788348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 590/1201] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:48:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:48:07.797935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09313ede-178e-4d00-ba70-ed873042ebf1 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:07.797935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09313ede-178e-4d00-ba70-ed873042ebf1 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:07.797935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09313ede-178e-4d00-ba70-ed873042ebf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:07.815534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-09313ede-178e-4d00-ba70-ed873042ebf1 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:07.816765 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-09313ede-178e-4d00-ba70-ed873042ebf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:07.902876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 612/1202] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:48:08.207589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e8740-2947-4f36-bcee-e432066fc924 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:08.340510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e8740-2947-4f36-bcee-e432066fc924 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:08.388525 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e287e-c7c7-4f69-8c48-4dc207b34b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:08.389384 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e287e-c7c7-4f69-8c48-4dc207b34b6f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:08.389784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e287e-c7c7-4f69-8c48-4dc207b34b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:08.390140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e287e-c7c7-4f69-8c48-4dc207b34b6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:08.405282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e287e-c7c7-4f69-8c48-4dc207b34b6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:08.407115 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e287e-c7c7-4f69-8c48-4dc207b34b6f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:08.427106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-3d2e287e-c7c7-4f69-8c48-4dc207b34b6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:08.427106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:08.427106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:08.427106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:08.427106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:08.427106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:08.427106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:08.427106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:08.427106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:08.427106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:08.427106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:08.427106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:08.427106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:08.427106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:08.427106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:08.427106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:08.428264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:08.428264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:08.428264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:08.428264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:08.428264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:08.428264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:08.428264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:08.428264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:08.428264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:08.428264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:08.428264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:08.428264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:08.428264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:08.428264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:08.428264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:08.428264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:08.428264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:08.429739 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:08.429739 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:08.429739 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:48:08.429739 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 591/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:08.435850 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09313ede-178e-4d00-ba70-ed873042ebf1 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:08.585196 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09313ede-178e-4d00-ba70-ed873042ebf1 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:08.625319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:08.634133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69c45e-4c94-4e23-820f-89b20390e171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:08.635059 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69c45e-4c94-4e23-820f-89b20390e171 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:08.635531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69c45e-4c94-4e23-820f-89b20390e171 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:08.635990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69c45e-4c94-4e23-820f-89b20390e171 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:08.636837 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69c45e-4c94-4e23-820f-89b20390e171 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:08.637147 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69c45e-4c94-4e23-820f-89b20390e171 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:08.638523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69c45e-4c94-4e23-820f-89b20390e171 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:08.655096 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 613/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:48:08.661926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e287e-c7c7-4f69-8c48-4dc207b34b6f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:08.737685 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:08.809404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e287e-c7c7-4f69-8c48-4dc207b34b6f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:08.851846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bce87a-51e9-42f4-89f0-39504c8d36b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:08.852618 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bce87a-51e9-42f4-89f0-39504c8d36b4 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:08.853063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bce87a-51e9-42f4-89f0-39504c8d36b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:08.853282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bce87a-51e9-42f4-89f0-39504c8d36b4 None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:08.854028 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bce87a-51e9-42f4-89f0-39504c8d36b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:08.855965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bce87a-51e9-42f4-89f0-39504c8d36b4 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:08.894774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 592/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Sep 22 14:48:08 2022] POST /identity/v3/users => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:48:11.384472 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655b7b2-a6d0-4095-8ae2-0b46a46ee4bf None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:11.384472 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655b7b2-a6d0-4095-8ae2-0b46a46ee4bf None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:11.386262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655b7b2-a6d0-4095-8ae2-0b46a46ee4bf None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:11.386602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 614/1206] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:48:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:48:11.392924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5bd01-9bb8-4c33-87ec-937b8791309a None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:11.393423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5bd01-9bb8-4c33-87ec-937b8791309a None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:11.393538 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5bd01-9bb8-4c33-87ec-937b8791309a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:11.420241 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-eaf5bd01-9bb8-4c33-87ec-937b8791309a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:11.421513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-eaf5bd01-9bb8-4c33-87ec-937b8791309a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:11.515765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 593/1207] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:48:11.782104 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655b7b2-a6d0-4095-8ae2-0b46a46ee4bf None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:11.904518 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655b7b2-a6d0-4095-8ae2-0b46a46ee4bf None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:11.943140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da011b-92b9-4874-8fde-cc74d2ee2533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:11.943921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da011b-92b9-4874-8fde-cc74d2ee2533 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:11.944206 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da011b-92b9-4874-8fde-cc74d2ee2533 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:11.944483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da011b-92b9-4874-8fde-cc74d2ee2533 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:11.958571 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da011b-92b9-4874-8fde-cc74d2ee2533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:48:11.964550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-20da011b-92b9-4874-8fde-cc74d2ee2533 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:48:11.964550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:11.964550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:11.964550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:11.964550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:11.964550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:11.964550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:11.964550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:11.964550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:11.964550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:11.964550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:11.964550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:11.964550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:48:11.964550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:48:11.964550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:48:11.964550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:48:11.966633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:48:11.966633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:48:11.966633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:48:11.966633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:48:11.966633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:11.966633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:11.966633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:11.966633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:11.966633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:11.966633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:11.966633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:11.966633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:11.966633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:11.966633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:11.966633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:11.966633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:11.967745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:11.967745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:11.967745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:11.967745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:11.967745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:48:11.967745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:48:11.967745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:48:11.967745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:48:11.967745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:48:11.967745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:48:11.967745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:48:11.967745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:11.967745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 615/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:48:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:11.973706 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5bd01-9bb8-4c33-87ec-937b8791309a None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:12.043500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:12.128678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5bd01-9bb8-4c33-87ec-937b8791309a None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:12.171028 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b7ab8-8420-4789-bb0f-609f729d49e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:12.171669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b7ab8-8420-4789-bb0f-609f729d49e8 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:12.171969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b7ab8-8420-4789-bb0f-609f729d49e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:12.172223 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b7ab8-8420-4789-bb0f-609f729d49e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:12.172901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b7ab8-8420-4789-bb0f-609f729d49e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:12.173202 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b7ab8-8420-4789-bb0f-609f729d49e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:12.174585 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b7ab8-8420-4789-bb0f-609f729d49e8 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:12.185706 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 594/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:48:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:12.192302 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da011b-92b9-4874-8fde-cc74d2ee2533 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:12.255988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:12.321773 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da011b-92b9-4874-8fde-cc74d2ee2533 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:12.369794 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b97fa-f43d-4d68-acd6-287e3700a749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:12.370449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b97fa-f43d-4d68-acd6-287e3700a749 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:12.370863 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b97fa-f43d-4d68-acd6-287e3700a749 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:12.371077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b97fa-f43d-4d68-acd6-287e3700a749 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:12.385275 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b97fa-f43d-4d68-acd6-287e3700a749 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:12.387172 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b97fa-f43d-4d68-acd6-287e3700a749 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:12.401911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-c55b97fa-f43d-4d68-acd6-287e3700a749 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:12.401911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:12.401911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:12.401911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:12.401911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:12.401911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:12.401911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:12.401911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:12.401911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:12.401911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:12.401911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:12.401911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:12.401911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:12.401911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:12.401911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:12.401911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:12.403053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:12.403053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:12.403053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:12.403053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:12.403053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:12.403053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:12.403053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:12.403053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:12.403053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:12.403053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:12.403053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:12.403053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:12.403053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:12.403053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:12.403053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:12.403053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:12.403053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:12.404047 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:12.404047 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:12.404047 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:12.404047 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 616/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:48:12.408867 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b7ab8-8420-4789-bb0f-609f729d49e8 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:12.533721 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:12.538228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b7ab8-8420-4789-bb0f-609f729d49e8 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:12.578711 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296a967-0f6a-4f9c-be42-c56478945f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:12.579364 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296a967-0f6a-4f9c-be42-c56478945f8d None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:12.579643 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296a967-0f6a-4f9c-be42-c56478945f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:12.579937 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296a967-0f6a-4f9c-be42-c56478945f8d None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:12.580559 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296a967-0f6a-4f9c-be42-c56478945f8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:12.580860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296a967-0f6a-4f9c-be42-c56478945f8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:12.582224 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296a967-0f6a-4f9c-be42-c56478945f8d None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:12.595017 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 595/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:12.601505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b97fa-f43d-4d68-acd6-287e3700a749 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:12.714864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b97fa-f43d-4d68-acd6-287e3700a749 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:12.716511 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:12.750972 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543f684-bb57-4be5-985a-7d263df7f201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:12.751650 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543f684-bb57-4be5-985a-7d263df7f201 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:12.751899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543f684-bb57-4be5-985a-7d263df7f201 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:12.752209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543f684-bb57-4be5-985a-7d263df7f201 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:12.760562 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6543f684-bb57-4be5-985a-7d263df7f201 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:12.761996 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6543f684-bb57-4be5-985a-7d263df7f201 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:12.773693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-6543f684-bb57-4be5-985a-7d263df7f201 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:12.773693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:12.773693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:12.773693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:12.773693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:12.773693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:12.773693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:12.773693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:12.773693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:12.773693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:12.773693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:12.773693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:12.773693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:48:12.773693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:48:12.773693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:48:12.773693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:12.775539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:12.776720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:12.776720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:12.776720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:48:12.776720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:48:12.776720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:48:12.776720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:48:12.776720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:48:12.776720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:48:12.776720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:12.776720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:12.781275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296a967-0f6a-4f9c-be42-c56478945f8d None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:12.781824 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 617/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:12.896451 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296a967-0f6a-4f9c-be42-c56478945f8d None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:12.935592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9429af2-5dc9-493f-88fb-306f27d95cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:12.936307 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9429af2-5dc9-493f-88fb-306f27d95cae None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:12.936655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9429af2-5dc9-493f-88fb-306f27d95cae None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:12.937004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9429af2-5dc9-493f-88fb-306f27d95cae None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:12.937786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9429af2-5dc9-493f-88fb-306f27d95cae None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:12.938166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9429af2-5dc9-493f-88fb-306f27d95cae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:12.939701 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9429af2-5dc9-493f-88fb-306f27d95cae None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:12.958673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 596/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Sep 22 14:48:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:12.964213 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543f684-bb57-4be5-985a-7d263df7f201 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:13.093044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543f684-bb57-4be5-985a-7d263df7f201 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:13.131063 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d12b53-3205-40c2-86e9-538db1e57cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:13.131675 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d12b53-3205-40c2-86e9-538db1e57cab None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:13.131948 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d12b53-3205-40c2-86e9-538db1e57cab None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:13.132219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d12b53-3205-40c2-86e9-538db1e57cab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:13.163413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:13.209416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d12b53-3205-40c2-86e9-538db1e57cab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:13.210938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d12b53-3205-40c2-86e9-538db1e57cab None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:13.349890 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 618/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:48:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:48:13.351356 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9429af2-5dc9-493f-88fb-306f27d95cae None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:13.372746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:13.480497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9429af2-5dc9-493f-88fb-306f27d95cae None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:13.520801 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f100aa2-eacb-4bf6-917c-c50e5a80f948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:13.521543 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f100aa2-eacb-4bf6-917c-c50e5a80f948 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:13.521882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f100aa2-eacb-4bf6-917c-c50e5a80f948 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:13.522157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f100aa2-eacb-4bf6-917c-c50e5a80f948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:13.535725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f100aa2-eacb-4bf6-917c-c50e5a80f948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:13.537272 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f100aa2-eacb-4bf6-917c-c50e5a80f948 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:13.558547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-0f100aa2-eacb-4bf6-917c-c50e5a80f948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:13.558547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:13.558547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:13.558547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:13.558547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:13.558547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:13.558547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:13.558547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:13.558547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:13.558547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:13.558547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:13.558547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:13.558547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:13.558547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:13.558547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:13.558547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:13.566040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:13.566040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:13.566040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:13.566040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:13.566040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:13.566040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:13.566040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:13.566040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:13.566040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:13.566040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:13.566040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:13.566040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:13.566040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:13.566040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:13.566040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:13.566040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:13.566040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:13.567201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:13.567201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:13.567201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:48:13.567201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 597/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:13.567509 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d12b53-3205-40c2-86e9-538db1e57cab None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:13.646380 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:13.700882 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d12b53-3205-40c2-86e9-538db1e57cab None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:13.752983 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210b648-7b83-42a2-af78-a64160f7ccfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:13.754129 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210b648-7b83-42a2-af78-a64160f7ccfc None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:13.754664 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210b648-7b83-42a2-af78-a64160f7ccfc None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:13.755971 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210b648-7b83-42a2-af78-a64160f7ccfc None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:13.756430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210b648-7b83-42a2-af78-a64160f7ccfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:13.756916 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210b648-7b83-42a2-af78-a64160f7ccfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:13.759801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9210b648-7b83-42a2-af78-a64160f7ccfc None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:13.772656 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 619/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:13.779412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f100aa2-eacb-4bf6-917c-c50e5a80f948 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:13.860203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:13.898916 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f100aa2-eacb-4bf6-917c-c50e5a80f948 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:13.938403 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb34398-f825-4fe0-98e3-71313f7c4955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:13.939202 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb34398-f825-4fe0-98e3-71313f7c4955 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:13.939523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb34398-f825-4fe0-98e3-71313f7c4955 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:13.939836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb34398-f825-4fe0-98e3-71313f7c4955 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:13.947722 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb34398-f825-4fe0-98e3-71313f7c4955 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:13.949441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb34398-f825-4fe0-98e3-71313f7c4955 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:13.961868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-0fb34398-f825-4fe0-98e3-71313f7c4955 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:13.961868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:13.961868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:13.961868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:13.961868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:13.961868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:13.961868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:13.961868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:13.961868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:13.961868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:13.961868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:13.961868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:13.961868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:48:13.961868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:48:13.961868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:48:13.962857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:48:13.962857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:13.962857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:13.962857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:13.962857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:13.962857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:13.962857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:13.962857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:13.962857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:13.962857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:13.962857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:13.962857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:13.962857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:13.962857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:13.962857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:13.962857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:13.962857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:13.963795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:48:13.963795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:48:13.963795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:48:13.963795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:48:13.963795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:48:13.963795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:48:13.963795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:13.963795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:48:13.963795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 598/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:48:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:13.969524 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210b648-7b83-42a2-af78-a64160f7ccfc None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:14.088416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210b648-7b83-42a2-af78-a64160f7ccfc None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:14.114959 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:14.128891 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ba1fa-e176-4d34-9fb9-51b602fddbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:14.129556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ba1fa-e176-4d34-9fb9-51b602fddbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:14.129910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ba1fa-e176-4d34-9fb9-51b602fddbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:14.130130 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ba1fa-e176-4d34-9fb9-51b602fddbb3 None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:14.130958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ba1fa-e176-4d34-9fb9-51b602fddbb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:14.131270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ba1fa-e176-4d34-9fb9-51b602fddbb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:14.132746 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ba1fa-e176-4d34-9fb9-51b602fddbb3 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:14.146660 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 620/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:48:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:14.153343 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb34398-f825-4fe0-98e3-71313f7c4955 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:14.267308 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb34398-f825-4fe0-98e3-71313f7c4955 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:14.297432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:14.304260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044af9ce-4b07-4cea-8568-16ed4a3c9acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:14.304866 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044af9ce-4b07-4cea-8568-16ed4a3c9acd None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:14.305146 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044af9ce-4b07-4cea-8568-16ed4a3c9acd None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:14.305973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044af9ce-4b07-4cea-8568-16ed4a3c9acd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:14.306714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044af9ce-4b07-4cea-8568-16ed4a3c9acd None admin] RBAC: Adding query filter params (role.id=35bdd5122aae45ebbe44158a433d2f99, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=47b6b34b4ee5492fb1f0699716e5118b) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:14.307046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044af9ce-4b07-4cea-8568-16ed4a3c9acd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:14.308490 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044af9ce-4b07-4cea-8568-16ed4a3c9acd None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:14.326766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 599/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:48:14 2022] GET /identity/v3/role_assignments?user.id=47b6b34b4ee5492fb1f0699716e5118b&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=35bdd5122aae45ebbe44158a433d2f99 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:16.476406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301e850-0123-448f-af3d-39208309acd2 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:16.476956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301e850-0123-448f-af3d-39208309acd2 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:16.476956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301e850-0123-448f-af3d-39208309acd2 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:16.478808 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 621/1220] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:48:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:48:16.484355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18fe33-6c0a-4a65-b9eb-ffb30bee1c47 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:16.484904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18fe33-6c0a-4a65-b9eb-ffb30bee1c47 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:16.484904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18fe33-6c0a-4a65-b9eb-ffb30bee1c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:16.506047 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-2b18fe33-6c0a-4a65-b9eb-ffb30bee1c47 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:16.507271 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2b18fe33-6c0a-4a65-b9eb-ffb30bee1c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:16.576842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 600/1221] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:48:16.849396 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301e850-0123-448f-af3d-39208309acd2 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:16.968421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301e850-0123-448f-af3d-39208309acd2 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:17.006829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd1a81-28a8-4cb5-b455-e2fb638b5cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:17.007477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd1a81-28a8-4cb5-b455-e2fb638b5cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:17.007732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd1a81-28a8-4cb5-b455-e2fb638b5cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:17.008016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd1a81-28a8-4cb5-b455-e2fb638b5cd2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:17.019333 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd1a81-28a8-4cb5-b455-e2fb638b5cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:48:17.024270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-54fd1a81-28a8-4cb5-b455-e2fb638b5cd2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:48:17.024270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:17.024270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:17.024270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:17.024270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:17.024270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:17.024270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:17.024270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:17.024270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:17.024270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:17.024270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:17.024270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:17.024270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:48:17.024270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:48:17.024270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:48:17.024270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:48:17.025636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:48:17.025636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:48:17.025636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:48:17.025636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:48:17.025636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:17.025636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:17.025636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:17.025636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:17.025636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:17.025636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:17.025636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:17.025636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:17.025636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:17.025636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:17.025636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:17.025636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:17.026689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:17.026689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:17.026689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:17.026689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:17.026689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:48:17.026689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:48:17.026689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:48:17.026689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:48:17.026689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:48:17.026689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:48:17.026689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:48:17.026689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:17.026689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 622/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:48:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:17.031551 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18fe33-6c0a-4a65-b9eb-ffb30bee1c47 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:17.146126 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18fe33-6c0a-4a65-b9eb-ffb30bee1c47 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:17.190658 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:17.205727 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ae614-b7c3-4198-bd59-030104f01863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:17.206341 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ae614-b7c3-4198-bd59-030104f01863 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:17.206618 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ae614-b7c3-4198-bd59-030104f01863 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:17.206914 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ae614-b7c3-4198-bd59-030104f01863 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:17.207587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ae614-b7c3-4198-bd59-030104f01863 None admin] RBAC: Adding query filter params (name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:17.207890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ae614-b7c3-4198-bd59-030104f01863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:17.209261 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ae614-b7c3-4198-bd59-030104f01863 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:17.219205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 601/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:48:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:48:17.225909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd1a81-28a8-4cb5-b455-e2fb638b5cd2 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:17.272949 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:17.339584 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd1a81-28a8-4cb5-b455-e2fb638b5cd2 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:17.378403 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5867b-8b7a-42dc-9ec2-37fb0101f649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:17.379068 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5867b-8b7a-42dc-9ec2-37fb0101f649 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:17.379341 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5867b-8b7a-42dc-9ec2-37fb0101f649 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:17.379612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5867b-8b7a-42dc-9ec2-37fb0101f649 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:17.390503 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5867b-8b7a-42dc-9ec2-37fb0101f649 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:17.392037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5867b-8b7a-42dc-9ec2-37fb0101f649 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:17.408146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-49a5867b-8b7a-42dc-9ec2-37fb0101f649 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:17.408146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:17.408146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:17.408146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:17.408146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:17.408146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:17.408146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:17.408146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:17.408146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:17.408146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:17.408146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:17.408146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:17.408146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:17.408146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:17.408146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:17.408146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:17.409612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:17.409612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:17.409612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:17.409612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:17.409612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:17.409612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:17.409612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:17.409612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:17.409612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:17.409612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:17.409612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:17.409612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:17.409612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:17.409612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:17.409612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:17.409612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:17.409612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:17.411120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:17.411120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:17.411120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:17.411120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 623/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:48:17.415548 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ae614-b7c3-4198-bd59-030104f01863 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:17.536997 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ae614-b7c3-4198-bd59-030104f01863 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:17.574207 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e23014-ef43-472a-a06d-75687a7bd48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:17.574841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e23014-ef43-472a-a06d-75687a7bd48b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:17.575116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e23014-ef43-472a-a06d-75687a7bd48b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:17.575383 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e23014-ef43-472a-a06d-75687a7bd48b None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:17.576017 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e23014-ef43-472a-a06d-75687a7bd48b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:17.576314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e23014-ef43-472a-a06d-75687a7bd48b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:17.577673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e23014-ef43-472a-a06d-75687a7bd48b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:17.591271 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 602/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:17.597529 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5867b-8b7a-42dc-9ec2-37fb0101f649 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:17.696441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:17.703318 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5867b-8b7a-42dc-9ec2-37fb0101f649 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:17.738792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9a9af-1a03-4b7b-bda5-8ae46daa6747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:17.739426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9a9af-1a03-4b7b-bda5-8ae46daa6747 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:17.739697 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9a9af-1a03-4b7b-bda5-8ae46daa6747 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:17.739965 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9a9af-1a03-4b7b-bda5-8ae46daa6747 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:17.747986 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9a9af-1a03-4b7b-bda5-8ae46daa6747 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:17.749426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9a9af-1a03-4b7b-bda5-8ae46daa6747 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:17.761161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-a1a9a9af-1a03-4b7b-bda5-8ae46daa6747 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:17.761161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:17.761161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:17.761161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:17.761161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:17.761161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:17.761161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:17.761161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:17.761161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:17.761161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:17.761161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:17.761161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:17.761161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:48:17.761161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:48:17.761161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:48:17.761161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:17.762400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:17.763735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:17.763735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:17.763735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:48:17.763735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:48:17.763735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:48:17.763735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:48:17.763735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:48:17.763735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:48:17.763735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:17.763735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:17.763735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 624/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:17.769587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e23014-ef43-472a-a06d-75687a7bd48b None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:17.827926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:17.890895 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e23014-ef43-472a-a06d-75687a7bd48b None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:17.927059 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd34961-0b70-44ec-be79-2b9a0aebcd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:17.927661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd34961-0b70-44ec-be79-2b9a0aebcd22 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:17.927932 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd34961-0b70-44ec-be79-2b9a0aebcd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:17.928195 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd34961-0b70-44ec-be79-2b9a0aebcd22 None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:17.928901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd34961-0b70-44ec-be79-2b9a0aebcd22 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:17.929198 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd34961-0b70-44ec-be79-2b9a0aebcd22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:17.931205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd34961-0b70-44ec-be79-2b9a0aebcd22 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:17.948412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 603/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Sep 22 14:48:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:17.954242 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9a9af-1a03-4b7b-bda5-8ae46daa6747 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:18.068377 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9a9af-1a03-4b7b-bda5-8ae46daa6747 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:18.105838 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18fbd2-a828-4bdb-8f9a-b3c22409d203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:18.106473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18fbd2-a828-4bdb-8f9a-b3c22409d203 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:18.106768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18fbd2-a828-4bdb-8f9a-b3c22409d203 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:18.107044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18fbd2-a828-4bdb-8f9a-b3c22409d203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:18.118667 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18fbd2-a828-4bdb-8f9a-b3c22409d203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:18.120161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18fbd2-a828-4bdb-8f9a-b3c22409d203 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:18.134676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-9b18fbd2-a828-4bdb-8f9a-b3c22409d203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:18.134676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:18.134676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:18.134676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:18.134676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:18.134676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:18.134676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:18.134676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:18.134676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:18.134676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:18.134676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:18.134676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:18.134676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:18.134676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:18.134676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:18.134676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:18.135715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:18.135715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:18.135715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:18.135715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:18.135715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:18.135715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:18.135715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:18.135715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:18.135715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:18.135715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:18.135715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:18.135715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:18.135715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:18.135715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:18.135715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:18.135715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:18.135715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:18.136617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:18.136617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:18.136617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:18.136617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 625/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:18.142124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd34961-0b70-44ec-be79-2b9a0aebcd22 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:18.251066 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd34961-0b70-44ec-be79-2b9a0aebcd22 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:18.253842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:18.290302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1e5da-101f-41d6-968c-2d74e3c4d4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:18.290947 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1e5da-101f-41d6-968c-2d74e3c4d4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:18.291279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1e5da-101f-41d6-968c-2d74e3c4d4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:18.291595 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1e5da-101f-41d6-968c-2d74e3c4d4c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:18.292232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1e5da-101f-41d6-968c-2d74e3c4d4c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:18.292580 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1e5da-101f-41d6-968c-2d74e3c4d4c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:18.294036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1e5da-101f-41d6-968c-2d74e3c4d4c6 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:18.307899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 604/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:18.313681 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18fbd2-a828-4bdb-8f9a-b3c22409d203 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:18.324966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:18.422221 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18fbd2-a828-4bdb-8f9a-b3c22409d203 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:18.457769 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f699b-5eb9-425e-8103-53cee3038304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:18.458442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f699b-5eb9-425e-8103-53cee3038304 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:18.458721 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f699b-5eb9-425e-8103-53cee3038304 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:18.459014 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f699b-5eb9-425e-8103-53cee3038304 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:18.466303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f699b-5eb9-425e-8103-53cee3038304 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:18.467778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f699b-5eb9-425e-8103-53cee3038304 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:18.480283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-395f699b-5eb9-425e-8103-53cee3038304 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:18.480283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:18.480283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:18.480283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:18.480283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:18.480283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:18.480283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:18.480283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:18.480283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:18.480283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:18.480283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:18.480283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:18.480283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:48:18.480283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:48:18.480283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:48:18.481500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:48:18.481500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:18.481500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:18.481500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:18.481500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:18.481500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:18.481500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:18.481500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:18.481500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:18.481500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:18.481500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:18.481500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:18.481500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:18.481500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:18.481500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:18.481500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:18.481500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:18.482418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:48:18.482418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:48:18.482418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:48:18.482418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:48:18.482418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:48:18.482418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:48:18.482418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:18.482418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:18.482418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 626/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:48:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:18.487695 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1e5da-101f-41d6-968c-2d74e3c4d4c6 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:18.594107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1e5da-101f-41d6-968c-2d74e3c4d4c6 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:18.630760 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b789ba-efd9-452f-8a0d-41c162ef0d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:18.631545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b789ba-efd9-452f-8a0d-41c162ef0d54 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:18.631822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b789ba-efd9-452f-8a0d-41c162ef0d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:18.632093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b789ba-efd9-452f-8a0d-41c162ef0d54 None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:18.632782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b789ba-efd9-452f-8a0d-41c162ef0d54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:18.633087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b789ba-efd9-452f-8a0d-41c162ef0d54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:18.635113 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b789ba-efd9-452f-8a0d-41c162ef0d54 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:18.647690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 605/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:48:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:48:18.654222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f699b-5eb9-425e-8103-53cee3038304 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:18.724924 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:18.765012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f699b-5eb9-425e-8103-53cee3038304 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:18.802315 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb1859-5ba1-4758-9928-2ff075a2000d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:18.803041 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb1859-5ba1-4758-9928-2ff075a2000d None admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:18.803324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb1859-5ba1-4758-9928-2ff075a2000d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:18.803605 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb1859-5ba1-4758-9928-2ff075a2000d None admin] PATH_INFO: `/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/47b6b34b4ee5492fb1f0699716e5118b/roles/35bdd5122aae45ebbe44158a433d2f99` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:18.819368 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb1859-5ba1-4758-9928-2ff075a2000d None admin] RBAC: Authorizing `identity:create_grant(project_id=43bc5871e4bf441b874474bcae4e5385, user_id=47b6b34b4ee5492fb1f0699716e5118b, role_id=35bdd5122aae45ebbe44158a433d2f99)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:18.820922 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb1859-5ba1-4758-9928-2ff075a2000d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:18.835201 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 627/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:48:18 2022] PUT /identity/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/47b6b34b4ee5492fb1f0699716e5118b/roles/35bdd5122aae45ebbe44158a433d2f99 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:48:21.021309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3008090-eef0-4d46-ab51-017dba38ddfd None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:21.021898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3008090-eef0-4d46-ab51-017dba38ddfd None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:21.021898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3008090-eef0-4d46-ab51-017dba38ddfd None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:21.023609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 606/1233] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:48:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:48:21.029199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689bf0f-5a3c-4107-b329-b470a85398c2 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:21.029780 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689bf0f-5a3c-4107-b329-b470a85398c2 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:21.029780 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689bf0f-5a3c-4107-b329-b470a85398c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:21.048252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-8689bf0f-5a3c-4107-b329-b470a85398c2 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:21.049441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-8689bf0f-5a3c-4107-b329-b470a85398c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:21.122570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 628/1234] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:48:21.400019 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3008090-eef0-4d46-ab51-017dba38ddfd None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:21.427723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:21.519172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3008090-eef0-4d46-ab51-017dba38ddfd None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:21.566506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567faf21-b643-4fb0-88f2-568ebd9a07fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:21.567171 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567faf21-b643-4fb0-88f2-568ebd9a07fe None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:21.567445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567faf21-b643-4fb0-88f2-568ebd9a07fe None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:21.567712 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567faf21-b643-4fb0-88f2-568ebd9a07fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:21.579618 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567faf21-b643-4fb0-88f2-568ebd9a07fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:48:21.585459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-567faf21-b643-4fb0-88f2-568ebd9a07fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:48:21.585459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:21.585459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:21.585459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:21.585459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:21.585459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:21.585459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:21.585459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:21.585459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:21.585459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:21.585459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:21.585459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:21.585459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:48:21.585459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:48:21.585459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:48:21.585459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:48:21.587137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:48:21.587137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:48:21.587137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:48:21.587137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:48:21.587137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:21.587137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:21.587137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:21.587137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:21.587137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:21.587137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:21.587137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:21.587137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:21.587137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:21.587137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:21.587137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:21.587137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:21.588132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:21.588132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:21.588132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:21.588132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:21.588132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:48:21.588132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:48:21.588132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:48:21.588132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:48:21.588132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:48:21.588132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:48:21.588132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 14:48:21.588132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:48:21.588132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 607/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Sep 22 14:48:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:48:21.594112 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689bf0f-5a3c-4107-b329-b470a85398c2 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:21.647385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:21.706923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689bf0f-5a3c-4107-b329-b470a85398c2 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:21.746208 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c520e-a711-428d-95fb-4a9d1ecda869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:21.746925 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c520e-a711-428d-95fb-4a9d1ecda869 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:21.747119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c520e-a711-428d-95fb-4a9d1ecda869 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:21.747400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c520e-a711-428d-95fb-4a9d1ecda869 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:21.748076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c520e-a711-428d-95fb-4a9d1ecda869 None admin] RBAC: Adding query filter params (name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:21.748350 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c520e-a711-428d-95fb-4a9d1ecda869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:21.749702 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c520e-a711-428d-95fb-4a9d1ecda869 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:21.758640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 629/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Sep 22 14:48:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:48:21.765689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567faf21-b643-4fb0-88f2-568ebd9a07fe None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:21.868364 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:21.886916 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567faf21-b643-4fb0-88f2-568ebd9a07fe None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:21.924513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2d550-19b5-486a-b634-e87e591e84be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:21.925211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2d550-19b5-486a-b634-e87e591e84be None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:21.925588 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2d550-19b5-486a-b634-e87e591e84be None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:21.925882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2d550-19b5-486a-b634-e87e591e84be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:21.939297 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2d550-19b5-486a-b634-e87e591e84be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:21.940775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2d550-19b5-486a-b634-e87e591e84be None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:21.956187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-2ad2d550-19b5-486a-b634-e87e591e84be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:21.956187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:21.956187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:21.956187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:21.956187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:21.956187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:21.956187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:21.956187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:21.956187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:21.956187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:21.956187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:21.956187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:21.956187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:21.956187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:21.956187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:21.956187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:21.957348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:21.957348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:21.957348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:21.957348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:21.957348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:21.957348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:21.957348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:21.957348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:21.957348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:21.957348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:21.957348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:21.957348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:21.957348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:21.957348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:21.957348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:21.957348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:21.957348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:21.958429 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:21.958429 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:21.958429 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:48:21.958429 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 608/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:21.964427 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c520e-a711-428d-95fb-4a9d1ecda869 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:22.079465 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c520e-a711-428d-95fb-4a9d1ecda869 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:22.090695 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:22.119553 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60eaeeb-60ee-4263-ade1-122a91a31480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:22.120093 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60eaeeb-60ee-4263-ade1-122a91a31480 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:22.120415 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60eaeeb-60ee-4263-ade1-122a91a31480 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:22.120634 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60eaeeb-60ee-4263-ade1-122a91a31480 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:22.121270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60eaeeb-60ee-4263-ade1-122a91a31480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:22.121569 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60eaeeb-60ee-4263-ade1-122a91a31480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:22.122958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60eaeeb-60ee-4263-ade1-122a91a31480 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:22.137061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 630/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:22.143662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2d550-19b5-486a-b634-e87e591e84be None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:22.262070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2d550-19b5-486a-b634-e87e591e84be None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:22.297512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f75cb-b0ab-4e0d-8bde-52e586e48a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:22.298140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f75cb-b0ab-4e0d-8bde-52e586e48a12 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:22.298412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f75cb-b0ab-4e0d-8bde-52e586e48a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:22.298678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f75cb-b0ab-4e0d-8bde-52e586e48a12 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:22.301462 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:22.308119 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f75cb-b0ab-4e0d-8bde-52e586e48a12 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:22.310075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f75cb-b0ab-4e0d-8bde-52e586e48a12 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:22.325845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-b17f75cb-b0ab-4e0d-8bde-52e586e48a12 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:22.325845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:22.325845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:22.325845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:22.325845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:22.325845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:22.325845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:22.325845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:22.325845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:22.325845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:22.325845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:22.325845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:22.325845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:48:22.325845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:48:22.325845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:48:22.325845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:22.327075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:22.328699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:22.328699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:22.328699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:48:22.328699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:48:22.328699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:48:22.328699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:48:22.328699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:48:22.328699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:48:22.328699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:22.328699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:48:22.328699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 609/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:22.334399 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60eaeeb-60ee-4263-ade1-122a91a31480 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:22.448565 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60eaeeb-60ee-4263-ade1-122a91a31480 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:22.490760 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8ff3b-4828-4f98-bc96-389b2075c884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:22.491381 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8ff3b-4828-4f98-bc96-389b2075c884 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:22.491657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8ff3b-4828-4f98-bc96-389b2075c884 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:22.491927 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8ff3b-4828-4f98-bc96-389b2075c884 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:22.493210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8ff3b-4828-4f98-bc96-389b2075c884 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:22.493512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8ff3b-4828-4f98-bc96-389b2075c884 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:22.495025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8ff3b-4828-4f98-bc96-389b2075c884 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:22.514710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 631/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Sep 22 14:48:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:22.521344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f75cb-b0ab-4e0d-8bde-52e586e48a12 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:22.640632 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f75cb-b0ab-4e0d-8bde-52e586e48a12 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:22.680531 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e6afe-653c-4795-878b-8d073fa4b006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:22.681158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e6afe-653c-4795-878b-8d073fa4b006 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:22.681445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e6afe-653c-4795-878b-8d073fa4b006 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:22.681695 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e6afe-653c-4795-878b-8d073fa4b006 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:22.748361 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:22.756000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e6afe-653c-4795-878b-8d073fa4b006 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:22.757470 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e6afe-653c-4795-878b-8d073fa4b006 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:22.875004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 610/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:48:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:48:22.881517 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8ff3b-4828-4f98-bc96-389b2075c884 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:22.911485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:22.999553 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8ff3b-4828-4f98-bc96-389b2075c884 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:23.047777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71931d4b-b73f-4834-989f-2990069b1b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:23.048521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71931d4b-b73f-4834-989f-2990069b1b40 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:23.048837 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71931d4b-b73f-4834-989f-2990069b1b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:23.049139 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71931d4b-b73f-4834-989f-2990069b1b40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:23.060384 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71931d4b-b73f-4834-989f-2990069b1b40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:23.061994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71931d4b-b73f-4834-989f-2990069b1b40 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:23.080024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-71931d4b-b73f-4834-989f-2990069b1b40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:23.080024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:23.080024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:23.080024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:23.080024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:23.080024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:23.080024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:23.080024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:23.080024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:23.080024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:23.080024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:23.080024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:23.080024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:23.080024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:23.080024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:23.080024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:23.081591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:23.081591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:23.081591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:23.081591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:23.081591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:23.081591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:23.081591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:23.081591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:23.081591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:23.081591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:23.081591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:23.081591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:23.081591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:23.081591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:23.081591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:23.081591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:23.081591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:23.082757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:23.082757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:23.082757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:23.087372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 632/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 14:48:23.088660 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e6afe-653c-4795-878b-8d073fa4b006 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:23.194876 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:23.201772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e6afe-653c-4795-878b-8d073fa4b006 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:23.243935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff569854-4ced-401a-a65b-a667965f96f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:23.244554 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff569854-4ced-401a-a65b-a667965f96f2 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:23.244832 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff569854-4ced-401a-a65b-a667965f96f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:23.245100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff569854-4ced-401a-a65b-a667965f96f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:23.245738 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff569854-4ced-401a-a65b-a667965f96f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:23.246040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff569854-4ced-401a-a65b-a667965f96f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:23.247544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff569854-4ced-401a-a65b-a667965f96f2 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:23.261663 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 611/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:48:23.267531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71931d4b-b73f-4834-989f-2990069b1b40 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:23.365904 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:23.393739 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71931d4b-b73f-4834-989f-2990069b1b40 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:23.430667 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5828d-00dd-475d-821e-37f3a59aa9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:23.431346 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5828d-00dd-475d-821e-37f3a59aa9b6 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:23.431625 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5828d-00dd-475d-821e-37f3a59aa9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:23.431902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5828d-00dd-475d-821e-37f3a59aa9b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:23.439324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5828d-00dd-475d-821e-37f3a59aa9b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:23.440770 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e5828d-00dd-475d-821e-37f3a59aa9b6 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:23.453991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-77e5828d-00dd-475d-821e-37f3a59aa9b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:23.453991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:23.453991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:23.453991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:23.453991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:23.453991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:23.453991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:23.453991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:23.453991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:23.453991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:23.453991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:23.453991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:23.453991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:48:23.453991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:48:23.453991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:48:23.455184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:48:23.455184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:23.455184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:23.455184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:23.455184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:23.455184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:23.455184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:23.455184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:23.455184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:23.455184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:23.455184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:23.455184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:23.455184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:23.455184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:23.455184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:23.455184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:23.455184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:23.456094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:48:23.456094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:48:23.456094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:48:23.456094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:48:23.456094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:48:23.456094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:48:23.456094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:23.456094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:23.456094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 633/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:48:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:23.462236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff569854-4ced-401a-a65b-a667965f96f2 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:23.581401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff569854-4ced-401a-a65b-a667965f96f2 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:23.618401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fd4c2-558d-4db5-a125-1d0e28fb7d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:23.619029 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fd4c2-558d-4db5-a125-1d0e28fb7d46 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:23.619304 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fd4c2-558d-4db5-a125-1d0e28fb7d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:23.619565 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fd4c2-558d-4db5-a125-1d0e28fb7d46 None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:23.620290 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079fd4c2-558d-4db5-a125-1d0e28fb7d46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:23.620589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079fd4c2-558d-4db5-a125-1d0e28fb7d46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:23.622054 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079fd4c2-558d-4db5-a125-1d0e28fb7d46 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:23.635334 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 612/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:48:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:23.641426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5828d-00dd-475d-821e-37f3a59aa9b6 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:23.756366 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e5828d-00dd-475d-821e-37f3a59aa9b6 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:23.795488 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb8e11-2228-4863-b6ca-76bb85ebd65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:23.796298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8e11-2228-4863-b6ca-76bb85ebd65d None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:23.796625 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8e11-2228-4863-b6ca-76bb85ebd65d None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:23.796926 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8e11-2228-4863-b6ca-76bb85ebd65d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:23.797832 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb8e11-2228-4863-b6ca-76bb85ebd65d None admin] RBAC: Adding query filter params (role.id=35bdd5122aae45ebbe44158a433d2f99, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=47b6b34b4ee5492fb1f0699716e5118b) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:23.798143 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb8e11-2228-4863-b6ca-76bb85ebd65d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:23.799617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb8e11-2228-4863-b6ca-76bb85ebd65d None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:23.806707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 634/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:48:23 2022] GET /identity/v3/role_assignments?user.id=47b6b34b4ee5492fb1f0699716e5118b&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=35bdd5122aae45ebbe44158a433d2f99 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:26.390830 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08811f6f-1758-4612-814a-e8e3dcf05de1 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:26.390830 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08811f6f-1758-4612-814a-e8e3dcf05de1 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:26.390830 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08811f6f-1758-4612-814a-e8e3dcf05de1 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:26.392858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 613/1247] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:48:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:48:26.399657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824a310-4ec9-4e2f-9ebd-7a94fcc43476 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:26.400461 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824a310-4ec9-4e2f-9ebd-7a94fcc43476 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:26.400632 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824a310-4ec9-4e2f-9ebd-7a94fcc43476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:26.409830 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:26.428011 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-3824a310-4ec9-4e2f-9ebd-7a94fcc43476 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:26.429774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3824a310-4ec9-4e2f-9ebd-7a94fcc43476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:26.514717 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 635/1248] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:48:26.789237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08811f6f-1758-4612-814a-e8e3dcf05de1 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:26.808854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:26.906084 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08811f6f-1758-4612-814a-e8e3dcf05de1 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:26.943575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed45dd5-05b1-4547-ae61-7398b2a32d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:26.944254 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed45dd5-05b1-4547-ae61-7398b2a32d55 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:26.944582 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed45dd5-05b1-4547-ae61-7398b2a32d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:26.944915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed45dd5-05b1-4547-ae61-7398b2a32d55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:26.956982 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed45dd5-05b1-4547-ae61-7398b2a32d55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:48:26.962330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-0ed45dd5-05b1-4547-ae61-7398b2a32d55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:48:26.962330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:26.962330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:26.962330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:26.962330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:26.962330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:26.962330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:26.962330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:26.962330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:26.962330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:26.962330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:26.962330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:26.962330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:48:26.962330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:48:26.962330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:48:26.962330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:48:26.963537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:48:26.963537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:48:26.963537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:48:26.963537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:48:26.963537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:26.963537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:26.963537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:26.963537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:26.963537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:26.963537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:26.963537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:26.963537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:26.963537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:26.963537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:26.963537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:26.963537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:26.964423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:26.964423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:26.964423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:26.964423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:26.964423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:48:26.964423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:48:26.964423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:48:26.964423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:48:26.964423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:48:26.964423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:48:26.964423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:48:26.964423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:48:26.964423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 614/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:48:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:26.969729 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824a310-4ec9-4e2f-9ebd-7a94fcc43476 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:27.085991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824a310-4ec9-4e2f-9ebd-7a94fcc43476 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:27.118333 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:27.123792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4df57a-eb08-4b8e-b296-392afc47942f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:27.124392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4df57a-eb08-4b8e-b296-392afc47942f None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:27.124665 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4df57a-eb08-4b8e-b296-392afc47942f None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:27.124932 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4df57a-eb08-4b8e-b296-392afc47942f None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:27.125582 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4df57a-eb08-4b8e-b296-392afc47942f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:27.125879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4df57a-eb08-4b8e-b296-392afc47942f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:27.127955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4df57a-eb08-4b8e-b296-392afc47942f None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:27.136160 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 636/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:48:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:48:27.142005 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed45dd5-05b1-4547-ae61-7398b2a32d55 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:27.227279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:27.257314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed45dd5-05b1-4547-ae61-7398b2a32d55 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:27.293888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d21345-a6a0-4ea7-9937-6e47708503b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:27.294512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d21345-a6a0-4ea7-9937-6e47708503b1 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:27.294807 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d21345-a6a0-4ea7-9937-6e47708503b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:27.295078 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d21345-a6a0-4ea7-9937-6e47708503b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:27.306386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d21345-a6a0-4ea7-9937-6e47708503b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:27.307900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d21345-a6a0-4ea7-9937-6e47708503b1 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:27.325191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-d3d21345-a6a0-4ea7-9937-6e47708503b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:27.325191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:27.325191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:27.325191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:27.325191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:27.325191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:27.325191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:27.325191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:27.325191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:27.325191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:27.325191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:27.325191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:27.325191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:27.325191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:27.325191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:27.325191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:27.326417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:27.326417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:27.326417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:27.326417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:27.326417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:27.326417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:27.326417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:27.326417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:27.326417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:27.326417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:27.326417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:27.326417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:27.326417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:27.326417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:27.326417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:27.326417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:27.326417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:27.327978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:27.327978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:27.327978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:48:27.327978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 615/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:27.332317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4df57a-eb08-4b8e-b296-392afc47942f None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:27.440589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4df57a-eb08-4b8e-b296-392afc47942f None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:27.478759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10d09b-9c4a-4979-8fb0-8a0def328d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:27.479366 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10d09b-9c4a-4979-8fb0-8a0def328d3b None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:27.479644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10d09b-9c4a-4979-8fb0-8a0def328d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:27.479919 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10d09b-9c4a-4979-8fb0-8a0def328d3b None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:27.480596 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10d09b-9c4a-4979-8fb0-8a0def328d3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:27.480950 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10d09b-9c4a-4979-8fb0-8a0def328d3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:27.482396 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10d09b-9c4a-4979-8fb0-8a0def328d3b None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:27.495116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 637/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:27.502165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d21345-a6a0-4ea7-9937-6e47708503b1 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:27.610904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d21345-a6a0-4ea7-9937-6e47708503b1 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:27.642111 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:27.651442 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41334d33-1b8c-4b90-a29f-f366527b162d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:27.652086 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41334d33-1b8c-4b90-a29f-f366527b162d None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:27.652366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41334d33-1b8c-4b90-a29f-f366527b162d None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:27.652971 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41334d33-1b8c-4b90-a29f-f366527b162d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:27.661865 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41334d33-1b8c-4b90-a29f-f366527b162d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:27.663371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41334d33-1b8c-4b90-a29f-f366527b162d None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:27.676858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-41334d33-1b8c-4b90-a29f-f366527b162d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:27.676858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:27.676858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:27.676858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:27.676858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:27.676858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:27.676858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:27.676858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:27.676858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:27.676858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:27.676858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:27.676858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:27.676858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:48:27.676858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:48:27.676858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:48:27.676858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:27.678036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:27.679228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:27.679228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:27.679228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:48:27.679228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:48:27.679228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:48:27.679228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:48:27.679228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:48:27.679228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:48:27.679228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:27.679228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:48:27.679228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 616/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:27.685024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10d09b-9c4a-4979-8fb0-8a0def328d3b None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:27.750506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:27.800064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10d09b-9c4a-4979-8fb0-8a0def328d3b None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:27.837011 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6000c-3b16-4c6a-b65c-d80bf6b3b527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:27.837639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6000c-3b16-4c6a-b65c-d80bf6b3b527 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:27.837919 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6000c-3b16-4c6a-b65c-d80bf6b3b527 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:27.838192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6000c-3b16-4c6a-b65c-d80bf6b3b527 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:27.838969 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6000c-3b16-4c6a-b65c-d80bf6b3b527 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:27.839276 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6000c-3b16-4c6a-b65c-d80bf6b3b527 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:27.840758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6000c-3b16-4c6a-b65c-d80bf6b3b527 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:27.861299 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 638/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Sep 22 14:48:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:27.867962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41334d33-1b8c-4b90-a29f-f366527b162d None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:27.987165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41334d33-1b8c-4b90-a29f-f366527b162d None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:28.031704 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1862c-d8f5-4769-9769-74d8e09dd647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:28.032238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1862c-d8f5-4769-9769-74d8e09dd647 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:28.032517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1862c-d8f5-4769-9769-74d8e09dd647 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:28.032791 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1862c-d8f5-4769-9769-74d8e09dd647 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:28.093468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:28.108149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1862c-d8f5-4769-9769-74d8e09dd647 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:28.109639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1862c-d8f5-4769-9769-74d8e09dd647 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:28.215134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 617/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:48:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 14:48:28.221259 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6000c-3b16-4c6a-b65c-d80bf6b3b527 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:28.325805 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6000c-3b16-4c6a-b65c-d80bf6b3b527 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:28.339932 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:28.361999 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea255060-4e33-47a4-a894-90d36681c383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:28.362635 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea255060-4e33-47a4-a894-90d36681c383 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:28.362948 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea255060-4e33-47a4-a894-90d36681c383 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:28.363222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea255060-4e33-47a4-a894-90d36681c383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:28.373933 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea255060-4e33-47a4-a894-90d36681c383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:28.375439 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea255060-4e33-47a4-a894-90d36681c383 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:28.391711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-ea255060-4e33-47a4-a894-90d36681c383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:28.391711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:28.391711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:28.391711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:28.391711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:28.391711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:28.391711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:28.391711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:28.391711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:28.391711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:28.391711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:28.391711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:28.391711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:28.391711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:28.391711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:28.391711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:28.392785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:28.392785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:28.392785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:28.392785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:28.392785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:28.392785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:28.392785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:28.392785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:28.392785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:28.392785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:28.392785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:28.392785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:28.392785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:28.392785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:28.392785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:28.392785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:28.392785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:28.393707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:28.393707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:28.393707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:28.393707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 639/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:28.399454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1862c-d8f5-4769-9769-74d8e09dd647 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:28.485372 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:28.497703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1862c-d8f5-4769-9769-74d8e09dd647 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:28.532037 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf6eb0-4516-4182-a675-a6cd87b88d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:28.532660 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf6eb0-4516-4182-a675-a6cd87b88d7f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:28.532903 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf6eb0-4516-4182-a675-a6cd87b88d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:28.533175 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf6eb0-4516-4182-a675-a6cd87b88d7f None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:28.533804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf6eb0-4516-4182-a675-a6cd87b88d7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:28.534107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf6eb0-4516-4182-a675-a6cd87b88d7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:28.535567 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf6eb0-4516-4182-a675-a6cd87b88d7f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:28.546587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 618/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:28.552322 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea255060-4e33-47a4-a894-90d36681c383 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:28.650525 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea255060-4e33-47a4-a894-90d36681c383 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:28.682561 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407e689-b13e-464e-9c92-c3187859bb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:28.683207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407e689-b13e-464e-9c92-c3187859bb96 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:28.683488 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407e689-b13e-464e-9c92-c3187859bb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:28.683760 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407e689-b13e-464e-9c92-c3187859bb96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:28.690210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407e689-b13e-464e-9c92-c3187859bb96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:28.691679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407e689-b13e-464e-9c92-c3187859bb96 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:28.703192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-1407e689-b13e-464e-9c92-c3187859bb96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:28.703192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:28.703192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:28.703192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:28.703192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:28.703192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:28.703192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:28.703192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:28.703192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:28.703192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:28.703192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:28.703192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:28.703192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:48:28.703192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:48:28.703192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:48:28.704236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:48:28.704236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:28.704236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:28.704236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:28.704236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:28.704236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:28.704236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:28.704236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:28.704236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:28.704236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:28.704236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:28.704236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:28.704236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:28.704236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:28.704236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:28.704236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:28.704236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:28.705166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:48:28.705166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:48:28.705166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:48:28.705166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:48:28.705166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:48:28.705166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:48:28.705166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:28.705166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:28.705166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 640/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:48:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:28.710124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf6eb0-4516-4182-a675-a6cd87b88d7f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:28.807679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf6eb0-4516-4182-a675-a6cd87b88d7f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:28.841040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de2f93-d830-4542-8c67-28335ebf478e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:28.841630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de2f93-d830-4542-8c67-28335ebf478e None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:28.841906 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de2f93-d830-4542-8c67-28335ebf478e None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:28.842174 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de2f93-d830-4542-8c67-28335ebf478e None admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:28.842874 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de2f93-d830-4542-8c67-28335ebf478e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:28.843177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de2f93-d830-4542-8c67-28335ebf478e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:28.844604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de2f93-d830-4542-8c67-28335ebf478e None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:28.857427 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 619/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:48:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:28.863508 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407e689-b13e-464e-9c92-c3187859bb96 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:28.889405 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:28.961197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407e689-b13e-464e-9c92-c3187859bb96 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:28.998410 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2e5be-89f2-44d3-a437-b9a0fdb39805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:28.999226 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2e5be-89f2-44d3-a437-b9a0fdb39805 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:28.999535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2e5be-89f2-44d3-a437-b9a0fdb39805 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:28.999875 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2e5be-89f2-44d3-a437-b9a0fdb39805 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:29.000739 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2e5be-89f2-44d3-a437-b9a0fdb39805 None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=47b6b34b4ee5492fb1f0699716e5118b) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:29.001094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2e5be-89f2-44d3-a437-b9a0fdb39805 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:29.003166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2e5be-89f2-44d3-a437-b9a0fdb39805 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:29.011271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 641/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:48:28 2022] GET /identity/v3/role_assignments?user.id=47b6b34b4ee5492fb1f0699716e5118b&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=32c8a81e339a462e973e76bb528ef35a => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:31.225174 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baa8da-47a9-4e47-9702-64d662817e69 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:31.225746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baa8da-47a9-4e47-9702-64d662817e69 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:31.225746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baa8da-47a9-4e47-9702-64d662817e69 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:31.227145 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 620/1261] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:48:31.232205 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38bcb8-4ce1-4c13-91ca-e8af6f3d07b8 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:31.232614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38bcb8-4ce1-4c13-91ca-e8af6f3d07b8 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:31.232798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38bcb8-4ce1-4c13-91ca-e8af6f3d07b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:31.251048 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-0a38bcb8-4ce1-4c13-91ca-e8af6f3d07b8 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:31.252235 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-0a38bcb8-4ce1-4c13-91ca-e8af6f3d07b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:31.318563 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 642/1262] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:48:31.603625 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8baa8da-47a9-4e47-9702-64d662817e69 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:31.614659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:31.705642 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8baa8da-47a9-4e47-9702-64d662817e69 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:31.740082 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5fdb0-a0f7-4287-8d0f-f711688e0d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:31.740703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5fdb0-a0f7-4287-8d0f-f711688e0d76 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:31.740975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5fdb0-a0f7-4287-8d0f-f711688e0d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:31.741239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5fdb0-a0f7-4287-8d0f-f711688e0d76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:31.751066 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5fdb0-a0f7-4287-8d0f-f711688e0d76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:48:31.755805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-91b5fdb0-a0f7-4287-8d0f-f711688e0d76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:48:31.755805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:31.755805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:31.755805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:31.755805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:31.755805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:31.755805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:31.755805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:31.755805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:31.755805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:31.755805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:31.755805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:31.755805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:48:31.755805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:48:31.755805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:48:31.755805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:48:31.756931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:48:31.756931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:48:31.756931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:48:31.756931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:48:31.756931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:31.756931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:31.756931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:31.756931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:31.756931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:31.756931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:31.756931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:31.756931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:31.756931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:31.756931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:31.756931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:31.756931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:31.758459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:31.758459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:31.758459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:31.758459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:31.758459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:48:31.758459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:48:31.758459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:48:31.758459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:48:31.758459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:48:31.758459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:48:31.758459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:48:31.758459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:48:31.758459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 621/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:48:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:31.763560 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38bcb8-4ce1-4c13-91ca-e8af6f3d07b8 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:31.785687 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:31.863278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38bcb8-4ce1-4c13-91ca-e8af6f3d07b8 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:31.897396 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b6b74-e3f0-4af3-b3ce-d42cca010874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:31.897998 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b6b74-e3f0-4af3-b3ce-d42cca010874 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:31.898306 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b6b74-e3f0-4af3-b3ce-d42cca010874 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:31.898578 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b6b74-e3f0-4af3-b3ce-d42cca010874 None admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:31.899265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b6b74-e3f0-4af3-b3ce-d42cca010874 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:31.899571 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b6b74-e3f0-4af3-b3ce-d42cca010874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:31.900939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289b6b74-e3f0-4af3-b3ce-d42cca010874 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:31.908809 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 643/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:48:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:31.914561 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5fdb0-a0f7-4287-8d0f-f711688e0d76 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:31.980197 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:32.017317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5fdb0-a0f7-4287-8d0f-f711688e0d76 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:32.056945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba45c93-2c63-413e-b2ef-e20b64528f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:32.057570 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba45c93-2c63-413e-b2ef-e20b64528f0a None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:32.057846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba45c93-2c63-413e-b2ef-e20b64528f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:32.058113 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba45c93-2c63-413e-b2ef-e20b64528f0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:32.068470 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba45c93-2c63-413e-b2ef-e20b64528f0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:32.069905 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba45c93-2c63-413e-b2ef-e20b64528f0a None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:32.084194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-bba45c93-2c63-413e-b2ef-e20b64528f0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:32.084194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:32.084194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:32.084194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:32.084194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:32.084194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:32.084194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:32.084194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:32.084194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:32.084194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:32.084194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:32.084194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:32.084194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:32.084194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:32.084194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:32.084194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:32.085431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:32.085431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:32.085431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:32.085431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:32.085431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:32.085431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:32.085431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:32.085431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:32.085431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:32.085431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:32.085431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:32.085431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:32.085431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:32.085431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:32.085431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:32.085431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:32.085431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:32.086432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:32.086432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:32.086432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:48:32.086432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 622/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:32.091411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b6b74-e3f0-4af3-b3ce-d42cca010874 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:32.170308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:32.189414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289b6b74-e3f0-4af3-b3ce-d42cca010874 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:32.223661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12acad-a131-4648-ae78-fa3719faff84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:32.224262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12acad-a131-4648-ae78-fa3719faff84 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:32.224542 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12acad-a131-4648-ae78-fa3719faff84 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:32.224813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12acad-a131-4648-ae78-fa3719faff84 None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:32.225448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12acad-a131-4648-ae78-fa3719faff84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:32.225747 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12acad-a131-4648-ae78-fa3719faff84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:32.227656 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12acad-a131-4648-ae78-fa3719faff84 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:32.239678 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 644/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:32.246033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba45c93-2c63-413e-b2ef-e20b64528f0a None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:32.347032 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba45c93-2c63-413e-b2ef-e20b64528f0a None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:32.370212 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:32.385482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6df834-d837-49c9-ab62-aadf7bef8a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:32.386109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6df834-d837-49c9-ab62-aadf7bef8a3b None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:32.386389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6df834-d837-49c9-ab62-aadf7bef8a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:32.386656 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6df834-d837-49c9-ab62-aadf7bef8a3b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:32.394335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6df834-d837-49c9-ab62-aadf7bef8a3b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:32.395810 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6df834-d837-49c9-ab62-aadf7bef8a3b None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:32.407792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-5a6df834-d837-49c9-ab62-aadf7bef8a3b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:32.407792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:32.407792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:32.407792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:32.407792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:32.407792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:32.407792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:32.407792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:32.407792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:32.407792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:32.407792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:32.407792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:32.407792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:48:32.407792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:48:32.407792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:48:32.407792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:32.408942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:32.410118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:32.410118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:32.410118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:48:32.410118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:48:32.410118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:48:32.410118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:48:32.410118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:48:32.410118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:48:32.410118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:32.410118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:48:32.410118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 623/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:32.414935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12acad-a131-4648-ae78-fa3719faff84 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:32.512905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12acad-a131-4648-ae78-fa3719faff84 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:32.545776 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c389f2-7b91-4c2a-90ab-4da5e3945e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:32.546367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c389f2-7b91-4c2a-90ab-4da5e3945e56 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:32.546645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c389f2-7b91-4c2a-90ab-4da5e3945e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:32.546940 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c389f2-7b91-4c2a-90ab-4da5e3945e56 None admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:32.547649 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c389f2-7b91-4c2a-90ab-4da5e3945e56 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:32.547998 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c389f2-7b91-4c2a-90ab-4da5e3945e56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:32.549392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c389f2-7b91-4c2a-90ab-4da5e3945e56 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:32.557831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:32.566392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 645/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Sep 22 14:48:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:32.572736 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6df834-d837-49c9-ab62-aadf7bef8a3b None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:32.670991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6df834-d837-49c9-ab62-aadf7bef8a3b None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:32.706759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ce86b-86a0-4b84-90c9-9e1822896ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:32.707411 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ce86b-86a0-4b84-90c9-9e1822896ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:32.707690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ce86b-86a0-4b84-90c9-9e1822896ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:32.707960 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ce86b-86a0-4b84-90c9-9e1822896ba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:32.718980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ce86b-86a0-4b84-90c9-9e1822896ba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:32.720692 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5ce86b-86a0-4b84-90c9-9e1822896ba7 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:32.734962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-9d5ce86b-86a0-4b84-90c9-9e1822896ba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:32.734962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:32.734962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:32.734962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:32.734962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:32.734962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:32.734962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:32.734962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:32.734962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:32.734962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:32.734962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:32.734962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:32.734962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:32.734962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:32.734962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:32.734962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:32.736101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:32.736101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:32.736101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:32.736101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:32.736101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:32.736101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:32.736101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:32.736101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:32.736101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:32.736101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:32.736101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:32.736101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:32.736101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:32.736101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:32.736101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:32.736101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:32.736101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:32.737041 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:32.737041 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:32.737041 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:48:32.737041 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 624/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:32.742385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c389f2-7b91-4c2a-90ab-4da5e3945e56 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:32.838412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c389f2-7b91-4c2a-90ab-4da5e3945e56 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:32.871451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7583bd-bebf-4c21-a804-fd482e89e4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:32.872045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7583bd-bebf-4c21-a804-fd482e89e4eb None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:32.872321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7583bd-bebf-4c21-a804-fd482e89e4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:32.872588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7583bd-bebf-4c21-a804-fd482e89e4eb None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:32.873216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7583bd-bebf-4c21-a804-fd482e89e4eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:32.873510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7583bd-bebf-4c21-a804-fd482e89e4eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:32.874902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7583bd-bebf-4c21-a804-fd482e89e4eb None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:32.887057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 646/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:48:32.894350 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ce86b-86a0-4b84-90c9-9e1822896ba7 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:32.925212 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:32.999354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5ce86b-86a0-4b84-90c9-9e1822896ba7 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:33.037323 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51ca14-3429-4092-8dd0-8f1426789698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:33.038304 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51ca14-3429-4092-8dd0-8f1426789698 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:33.039318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51ca14-3429-4092-8dd0-8f1426789698 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:33.039795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51ca14-3429-4092-8dd0-8f1426789698 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:33.049398 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51ca14-3429-4092-8dd0-8f1426789698 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:33.050898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51ca14-3429-4092-8dd0-8f1426789698 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:33.063886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-db51ca14-3429-4092-8dd0-8f1426789698 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:33.063886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:33.063886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:33.063886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:33.063886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:33.063886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:33.063886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:33.063886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:33.063886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:33.063886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:33.063886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:33.063886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:33.063886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:48:33.063886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:48:33.063886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:48:33.064975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:48:33.064975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:33.064975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:33.064975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:33.064975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:33.064975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:33.064975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:33.064975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:33.064975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:33.064975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:33.064975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:33.064975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:33.064975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:33.064975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:33.064975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:33.064975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:33.064975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:33.065946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:48:33.065946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:48:33.065946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:48:33.065946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:48:33.065946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:48:33.065946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:48:33.065946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:33.065946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:48:33.065946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 625/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:48:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:33.071411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7583bd-bebf-4c21-a804-fd482e89e4eb None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:33.130574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:33.171646 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7583bd-bebf-4c21-a804-fd482e89e4eb None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:33.204248 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27323cb1-d5b2-4612-b67c-22ade163e2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:33.204843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27323cb1-d5b2-4612-b67c-22ade163e2d9 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:33.205122 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27323cb1-d5b2-4612-b67c-22ade163e2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:33.205395 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27323cb1-d5b2-4612-b67c-22ade163e2d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:33.206078 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27323cb1-d5b2-4612-b67c-22ade163e2d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:33.206383 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27323cb1-d5b2-4612-b67c-22ade163e2d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:33.207947 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27323cb1-d5b2-4612-b67c-22ade163e2d9 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:33.219680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 647/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:48:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:48:33.225918 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51ca14-3429-4092-8dd0-8f1426789698 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:33.315285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:33.324995 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51ca14-3429-4092-8dd0-8f1426789698 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:33.361989 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a3605-4908-420e-b531-e77df83346d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:33.362653 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a3605-4908-420e-b531-e77df83346d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:33.362953 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a3605-4908-420e-b531-e77df83346d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:33.363221 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a3605-4908-420e-b531-e77df83346d3 None admin] PATH_INFO: `/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/47b6b34b4ee5492fb1f0699716e5118b/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:33.365097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a3605-4908-420e-b531-e77df83346d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=43bc5871e4bf441b874474bcae4e5385, user_id=47b6b34b4ee5492fb1f0699716e5118b, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:33.366518 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a3605-4908-420e-b531-e77df83346d3 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:33.379379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 626/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Sep 22 14:48:33 2022] PUT /identity/v3/projects/43bc5871e4bf441b874474bcae4e5385/users/47b6b34b4ee5492fb1f0699716e5118b/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:48:35.471745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad9326-9e5d-49ea-9176-558bb47fdc36 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:35.472274 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad9326-9e5d-49ea-9176-558bb47fdc36 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:35.472669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad9326-9e5d-49ea-9176-558bb47fdc36 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:35.474011 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 648/1274] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:48:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:48:35.480077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35df35a-68cb-4049-bc0f-b94fe9ec582b None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:35.480518 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35df35a-68cb-4049-bc0f-b94fe9ec582b None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:35.480607 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35df35a-68cb-4049-bc0f-b94fe9ec582b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:35.498641 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-d35df35a-68cb-4049-bc0f-b94fe9ec582b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:35.499885 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d35df35a-68cb-4049-bc0f-b94fe9ec582b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:35.565673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 627/1275] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:48:35.834557 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad9326-9e5d-49ea-9176-558bb47fdc36 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:35.933967 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad9326-9e5d-49ea-9176-558bb47fdc36 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:35.951151 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:35.968049 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ff5f5-0b88-4a54-87fe-7749927c6148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:35.968677 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ff5f5-0b88-4a54-87fe-7749927c6148 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:35.968955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ff5f5-0b88-4a54-87fe-7749927c6148 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:35.969229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ff5f5-0b88-4a54-87fe-7749927c6148 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:35.983280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ff5f5-0b88-4a54-87fe-7749927c6148 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 14:48:35.989680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-de1ff5f5-0b88-4a54-87fe-7749927c6148 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:48:35.989680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:35.989680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:35.989680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:35.989680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:35.989680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:35.989680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:35.989680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:35.989680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:35.989680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:35.989680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:35.989680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:35.989680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 14:48:35.989680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 14:48:35.989680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 14:48:35.989680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 14:48:35.990984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 14:48:35.990984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 14:48:35.990984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 14:48:35.990984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 14:48:35.990984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:35.990984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:35.990984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:35.990984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:35.990984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:35.990984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:35.990984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:35.990984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:35.990984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:35.990984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:35.990984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:35.990984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:35.992745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:35.992745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:35.992745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:35.992745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:35.992745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 14:48:35.992745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 14:48:35.992745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 14:48:35.992745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 14:48:35.992745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 14:48:35.992745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 14:48:35.992745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 14:48:35.992745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:35.992745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 649/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Sep 22 14:48:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:35.998954 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35df35a-68cb-4049-bc0f-b94fe9ec582b None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:36.091464 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:36.114438 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35df35a-68cb-4049-bc0f-b94fe9ec582b None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:36.150088 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77500494-d1a3-4001-9766-d03d38bfae78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:36.150593 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77500494-d1a3-4001-9766-d03d38bfae78 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:36.150908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77500494-d1a3-4001-9766-d03d38bfae78 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:36.151178 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77500494-d1a3-4001-9766-d03d38bfae78 None admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:36.151839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77500494-d1a3-4001-9766-d03d38bfae78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:36.152142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77500494-d1a3-4001-9766-d03d38bfae78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:36.153512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77500494-d1a3-4001-9766-d03d38bfae78 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:36.161576 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 628/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Sep 22 14:48:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:36.168054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ff5f5-0b88-4a54-87fe-7749927c6148 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:36.269772 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ff5f5-0b88-4a54-87fe-7749927c6148 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:36.303240 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858522c-c313-4806-bf0f-92e69e13d149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:36.303864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858522c-c313-4806-bf0f-92e69e13d149 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:36.304140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858522c-c313-4806-bf0f-92e69e13d149 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:36.304946 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858522c-c313-4806-bf0f-92e69e13d149 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:36.315094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858522c-c313-4806-bf0f-92e69e13d149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:36.316557 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858522c-c313-4806-bf0f-92e69e13d149 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:36.331007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-c858522c-c313-4806-bf0f-92e69e13d149 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:36.331007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:36.331007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:36.331007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:36.331007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:36.331007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:36.331007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:36.331007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:36.331007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:36.331007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:36.331007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:36.331007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:36.331007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:36.331007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:36.331007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:36.331007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:36.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:36.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:36.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:36.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:36.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:36.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:36.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:36.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:36.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:36.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:36.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:36.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:36.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:36.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:36.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:36.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:36.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:36.333185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:36.333185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:36.333185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:36.333185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 650/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:36.338506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77500494-d1a3-4001-9766-d03d38bfae78 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:36.438945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77500494-d1a3-4001-9766-d03d38bfae78 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:36.471982 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d90973-943a-46c3-a640-f9e8d2c15f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:36.472575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d90973-943a-46c3-a640-f9e8d2c15f63 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:36.472858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d90973-943a-46c3-a640-f9e8d2c15f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:36.473129 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d90973-943a-46c3-a640-f9e8d2c15f63 None admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:36.473762 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d90973-943a-46c3-a640-f9e8d2c15f63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:36.474063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d90973-943a-46c3-a640-f9e8d2c15f63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:36.475456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d90973-943a-46c3-a640-f9e8d2c15f63 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:36.487555 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 629/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:36.493355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858522c-c313-4806-bf0f-92e69e13d149 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:36.522832 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:36.592010 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858522c-c313-4806-bf0f-92e69e13d149 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:36.625189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97599e-a239-407a-82f2-195685c64aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:36.625810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97599e-a239-407a-82f2-195685c64aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:36.626082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97599e-a239-407a-82f2-195685c64aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:36.626350 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97599e-a239-407a-82f2-195685c64aa3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:36.633538 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97599e-a239-407a-82f2-195685c64aa3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:36.634977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97599e-a239-407a-82f2-195685c64aa3 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:36.645778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-ab97599e-a239-407a-82f2-195685c64aa3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:36.645778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:36.645778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:36.645778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:36.645778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:36.645778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:36.645778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:36.645778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:36.645778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:36.645778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:36.645778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:36.645778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:36.645778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 14:48:36.645778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 14:48:36.645778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 14:48:36.645778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:36.647037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:36.648868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:36.648868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:36.648868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 14:48:36.648868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 14:48:36.648868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 14:48:36.648868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 14:48:36.648868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 14:48:36.648868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 14:48:36.648868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 14:48:36.648868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:36.648868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 651/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:36.653323 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d90973-943a-46c3-a640-f9e8d2c15f63 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:36.657063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:36.751981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d90973-943a-46c3-a640-f9e8d2c15f63 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:36.785502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a95c9-e34c-4405-b760-b22130751221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:36.786099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a95c9-e34c-4405-b760-b22130751221 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:36.786375 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a95c9-e34c-4405-b760-b22130751221 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:36.786643 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a95c9-e34c-4405-b760-b22130751221 None admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:36.787386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a95c9-e34c-4405-b760-b22130751221 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:36.787682 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a95c9-e34c-4405-b760-b22130751221 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:36.789164 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a95c9-e34c-4405-b760-b22130751221 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:36.806617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 630/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Sep 22 14:48:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:36.812406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97599e-a239-407a-82f2-195685c64aa3 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:36.904580 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:36.914445 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97599e-a239-407a-82f2-195685c64aa3 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:36.947853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2807e9d-1874-490b-bbfd-160cc1f7fbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:36.948445 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2807e9d-1874-490b-bbfd-160cc1f7fbb6 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:36.948720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2807e9d-1874-490b-bbfd-160cc1f7fbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:36.948990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2807e9d-1874-490b-bbfd-160cc1f7fbb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:37.018298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2807e9d-1874-490b-bbfd-160cc1f7fbb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:37.019675 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2807e9d-1874-490b-bbfd-160cc1f7fbb6 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:37.104612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:37.114496 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 652/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Sep 22 14:48:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 14:48:37.120902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a95c9-e34c-4405-b760-b22130751221 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:37.180870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:37.218786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a95c9-e34c-4405-b760-b22130751221 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:37.251996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55be5b-01cf-488f-adaa-5e13bc421e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:37.252616 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55be5b-01cf-488f-adaa-5e13bc421e6f None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:37.252891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55be5b-01cf-488f-adaa-5e13bc421e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:37.253157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55be5b-01cf-488f-adaa-5e13bc421e6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:37.263770 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55be5b-01cf-488f-adaa-5e13bc421e6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:37.265217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55be5b-01cf-488f-adaa-5e13bc421e6f None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:37.279689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-df55be5b-01cf-488f-adaa-5e13bc421e6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:37.279689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:37.279689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:37.279689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:37.279689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:37.279689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:37.279689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:37.279689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:37.279689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:37.279689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:37.279689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:37.279689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:37.279689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 14:48:37.279689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 14:48:37.279689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 14:48:37.279689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 14:48:37.280822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:37.280822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:37.280822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:37.280822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:37.280822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:37.280822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:37.280822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:37.280822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:37.280822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:37.280822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:37.280822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:37.280822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:37.280822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:37.280822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:37.280822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:37.280822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:37.280822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 14:48:37.282089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 14:48:37.282089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 14:48:37.282089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:48:37.282089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 631/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:48:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:37.286510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2807e9d-1874-490b-bbfd-160cc1f7fbb6 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:37.384213 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2807e9d-1874-490b-bbfd-160cc1f7fbb6 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:37.417134 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25938d-6f31-4673-9da3-033b2dbab1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:37.417736 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25938d-6f31-4673-9da3-033b2dbab1ed None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:37.418010 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25938d-6f31-4673-9da3-033b2dbab1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:37.418323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25938d-6f31-4673-9da3-033b2dbab1ed None admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:37.420100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25938d-6f31-4673-9da3-033b2dbab1ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:37.420465 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25938d-6f31-4673-9da3-033b2dbab1ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:37.421828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25938d-6f31-4673-9da3-033b2dbab1ed None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:37.435754 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 653/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Sep 22 14:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:37.441420 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55be5b-01cf-488f-adaa-5e13bc421e6f None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:37.543390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55be5b-01cf-488f-adaa-5e13bc421e6f None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:37.558886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:37.578971 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b0cab-f0bb-4772-ac94-63fda579ccc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:37.579602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b0cab-f0bb-4772-ac94-63fda579ccc5 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:37.579875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b0cab-f0bb-4772-ac94-63fda579ccc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:37.580143 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b0cab-f0bb-4772-ac94-63fda579ccc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:37.586620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b0cab-f0bb-4772-ac94-63fda579ccc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:37.588089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b0cab-f0bb-4772-ac94-63fda579ccc5 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:37.599345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application [None req-070b0cab-f0bb-4772-ac94-63fda579ccc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:37.599345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:37.599345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:37.599345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:37.599345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:37.599345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:37.599345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:37.599345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:37.599345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:37.599345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:37.599345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:37.599345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:37.599345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 14:48:37.599345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 14:48:37.599345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 14:48:37.600524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 14:48:37.600524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:37.600524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:37.600524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:37.600524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:37.600524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:37.600524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:37.600524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:37.600524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:37.600524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:37.600524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:37.600524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:37.600524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:37.600524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:37.600524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:37.600524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:37.600524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:37.601915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 14:48:37.601915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 14:48:37.601915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 14:48:37.601915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 14:48:37.601915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 14:48:37.601915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 14:48:37.601915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 14:48:37.601915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: ERROR keystone.server.flask.application Sep 22 14:48:37.601915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 632/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Sep 22 14:48:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:37.607438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25938d-6f31-4673-9da3-033b2dbab1ed None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:37.656301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:37.713600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25938d-6f31-4673-9da3-033b2dbab1ed None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:37.750172 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c2adb-7480-4e36-9688-9d0bf75c3b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:37.750822 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c2adb-7480-4e36-9688-9d0bf75c3b91 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:37.751096 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c2adb-7480-4e36-9688-9d0bf75c3b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:37.751369 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c2adb-7480-4e36-9688-9d0bf75c3b91 None admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:37.752054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c2adb-7480-4e36-9688-9d0bf75c3b91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:37.752358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c2adb-7480-4e36-9688-9d0bf75c3b91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:37.753791 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c2adb-7480-4e36-9688-9d0bf75c3b91 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:37.765809 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 654/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Sep 22 14:48:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:37.772020 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b0cab-f0bb-4772-ac94-63fda579ccc5 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:37.871520 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b0cab-f0bb-4772-ac94-63fda579ccc5 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:37.904863 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ea107-4dea-4b67-97a4-02b76f772222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:37.905450 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ea107-4dea-4b67-97a4-02b76f772222 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:37.905743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ea107-4dea-4b67-97a4-02b76f772222 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:37.906008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ea107-4dea-4b67-97a4-02b76f772222 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:37.906715 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ea107-4dea-4b67-97a4-02b76f772222 None admin] RBAC: Adding query filter params (role.id=32c8a81e339a462e973e76bb528ef35a, scope.project.id=43bc5871e4bf441b874474bcae4e5385, user.id=47b6b34b4ee5492fb1f0699716e5118b) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:37.907048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ea107-4dea-4b67-97a4-02b76f772222 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:37.908454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ea107-4dea-4b67-97a4-02b76f772222 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:37.914617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 633/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Sep 22 14:48:37 2022] GET /identity/v3/role_assignments?user.id=47b6b34b4ee5492fb1f0699716e5118b&scope.project.id=43bc5871e4bf441b874474bcae4e5385&role.id=32c8a81e339a462e973e76bb528ef35a => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:48:40.094570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef0939-62a7-4afc-a461-9a812facc0a4 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:40.095141 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef0939-62a7-4afc-a461-9a812facc0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:40.095141 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef0939-62a7-4afc-a461-9a812facc0a4 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:40.096744 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 655/1288] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:48:40.101935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50093f46-3f2e-4a96-a90f-9cd45b62c5c4 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:40.102385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50093f46-3f2e-4a96-a90f-9cd45b62c5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:40.102487 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50093f46-3f2e-4a96-a90f-9cd45b62c5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:40.121016 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-50093f46-3f2e-4a96-a90f-9cd45b62c5c4 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:40.122197 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-50093f46-3f2e-4a96-a90f-9cd45b62c5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:40.151273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:40.188038 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 634/1289] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 14:48:40.457183 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef0939-62a7-4afc-a461-9a812facc0a4 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:40.559886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef0939-62a7-4afc-a461-9a812facc0a4 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:40.570669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:40.593674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70d2c9-40b2-4cd1-95f0-8cc820d24967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:40.594303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70d2c9-40b2-4cd1-95f0-8cc820d24967 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:40.594581 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70d2c9-40b2-4cd1-95f0-8cc820d24967 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:40.594889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70d2c9-40b2-4cd1-95f0-8cc820d24967 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:40.595723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70d2c9-40b2-4cd1-95f0-8cc820d24967 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:40.597077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70d2c9-40b2-4cd1-95f0-8cc820d24967 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:40.606229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-6c70d2c9-40b2-4cd1-95f0-8cc820d24967 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:40.606229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:40.606229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:48:40.606229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:48:40.606229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:48:40.606229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:48:40.606229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:48:40.606229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:48:40.606229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:40.606229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:40.606229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:48:40.606229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:40.606229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:40.606229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:40.606229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:40.606229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:40.606229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:40.607546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:40.607546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:40.607546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:40.607546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:40.607546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:40.607546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:40.607546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:48:40.607546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:48:40.607546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:48:40.607546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:48:40.607546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:40.607546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:40.607546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:40.607546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:40.607546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:40.607546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:40.608443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:40.608443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:40.608443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:40.608443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:40.608443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:40.608443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:40.608443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:40.608443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:40.608443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:40.608443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:40.608443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:48:40.608443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:48:40.608443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:40.608443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:40.608443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 656/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Sep 22 14:48:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:40.613452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50093f46-3f2e-4a96-a90f-9cd45b62c5c4 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:40.709037 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50093f46-3f2e-4a96-a90f-9cd45b62c5c4 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:40.743688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67530156-928d-45d4-899e-fd1bedeadcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:40.744283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67530156-928d-45d4-899e-fd1bedeadcd5 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:40.744555 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67530156-928d-45d4-899e-fd1bedeadcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:40.744820 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67530156-928d-45d4-899e-fd1bedeadcd5 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:40.745428 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67530156-928d-45d4-899e-fd1bedeadcd5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:40.745724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67530156-928d-45d4-899e-fd1bedeadcd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:40.747116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67530156-928d-45d4-899e-fd1bedeadcd5 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:40.751852 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 635/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Thu Sep 22 14:48:40 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:40.757913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70d2c9-40b2-4cd1-95f0-8cc820d24967 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:40.855222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70d2c9-40b2-4cd1-95f0-8cc820d24967 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:40.887832 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66854cdb-6512-41b4-a4d4-927de6fe35ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:40.888433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66854cdb-6512-41b4-a4d4-927de6fe35ea None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:40.888710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66854cdb-6512-41b4-a4d4-927de6fe35ea None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:40.888982 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66854cdb-6512-41b4-a4d4-927de6fe35ea None admin] PATH_INFO: `/v3/services` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:40.889602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66854cdb-6512-41b4-a4d4-927de6fe35ea None admin] RBAC: Adding query filter params (type=placement) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:40.889904 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66854cdb-6512-41b4-a4d4-927de6fe35ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:40.891291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66854cdb-6512-41b4-a4d4-927de6fe35ea None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:40.895658 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 657/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Thu Sep 22 14:48:40 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:43.049909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe9e58-eb12-4b7d-95dc-4dacb681fda8 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:43.050438 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe9e58-eb12-4b7d-95dc-4dacb681fda8 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:43.050438 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe9e58-eb12-4b7d-95dc-4dacb681fda8 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:43.051831 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 636/1293] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:48:43.057235 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f834b68-d414-49a3-a062-07c9eb33aae1 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:43.057637 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f834b68-d414-49a3-a062-07c9eb33aae1 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:43.057795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f834b68-d414-49a3-a062-07c9eb33aae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:43.076111 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-2f834b68-d414-49a3-a062-07c9eb33aae1 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:43.077318 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-2f834b68-d414-49a3-a062-07c9eb33aae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:43.108178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:43.145882 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 658/1294] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:48:43.410015 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe9e58-eb12-4b7d-95dc-4dacb681fda8 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:43.440328 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:43.511185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe9e58-eb12-4b7d-95dc-4dacb681fda8 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:43.543894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15d25f-3c69-4230-b365-4fa69c572cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:43.544490 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15d25f-3c69-4230-b365-4fa69c572cb9 None admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:43.544765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15d25f-3c69-4230-b365-4fa69c572cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:43.545034 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15d25f-3c69-4230-b365-4fa69c572cb9 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:43.545675 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15d25f-3c69-4230-b365-4fa69c572cb9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:43.547086 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15d25f-3c69-4230-b365-4fa69c572cb9 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:43.561159 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 637/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Sep 22 14:48:43 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:48:45.743851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891c6e2-43b8-4ce9-be4c-800693fb659d None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:45.744364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891c6e2-43b8-4ce9-be4c-800693fb659d None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:45.744364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891c6e2-43b8-4ce9-be4c-800693fb659d None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:45.745489 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 659/1296] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:48:45.750786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c7f1b-72ab-4b7b-8f98-f550f7b5228c None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:45.751175 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c7f1b-72ab-4b7b-8f98-f550f7b5228c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:45.751323 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c7f1b-72ab-4b7b-8f98-f550f7b5228c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:45.771166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-231c7f1b-72ab-4b7b-8f98-f550f7b5228c None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:45.772355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-231c7f1b-72ab-4b7b-8f98-f550f7b5228c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:45.846029 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 638/1297] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:48:46.111014 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891c6e2-43b8-4ce9-be4c-800693fb659d None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:46.215934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891c6e2-43b8-4ce9-be4c-800693fb659d None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:46.248821 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e66a9a-9c96-4503-9982-2aa351032022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:46.249439 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e66a9a-9c96-4503-9982-2aa351032022 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:46.249714 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e66a9a-9c96-4503-9982-2aa351032022 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:46.249982 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e66a9a-9c96-4503-9982-2aa351032022 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:46.250549 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e66a9a-9c96-4503-9982-2aa351032022 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:46.251932 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e66a9a-9c96-4503-9982-2aa351032022 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:46.260416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-a4e66a9a-9c96-4503-9982-2aa351032022 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:46.260416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:46.260416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:48:46.260416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:48:46.260416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:48:46.260416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:48:46.260416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:48:46.260416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:48:46.260416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:46.260416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:46.260416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:48:46.260416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:46.260416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:46.260416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:46.260416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:46.260416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:46.260416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:46.261689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:46.261689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:46.261689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:46.261689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:46.261689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:46.261689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:46.261689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:48:46.261689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:48:46.261689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:48:46.261689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:48:46.261689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:46.261689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:46.261689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:46.261689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:46.261689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:46.261689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:46.262855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:46.262855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:46.262855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:46.262855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:46.262855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:46.262855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:46.262855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:46.262855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:46.262855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:46.262855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:46.262855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:48:46.262855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:48:46.262855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:46.262855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:46.262855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 660/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Sep 22 14:48:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:46.267590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c7f1b-72ab-4b7b-8f98-f550f7b5228c None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:46.286078 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:46.368549 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c7f1b-72ab-4b7b-8f98-f550f7b5228c None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:46.402478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc443184-c94c-4ef7-aa57-971548ca8b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:46.403109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc443184-c94c-4ef7-aa57-971548ca8b65 None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:46.403389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc443184-c94c-4ef7-aa57-971548ca8b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:46.403660 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc443184-c94c-4ef7-aa57-971548ca8b65 None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:46.404282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc443184-c94c-4ef7-aa57-971548ca8b65 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:46.404585 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc443184-c94c-4ef7-aa57-971548ca8b65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:46.405944 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc443184-c94c-4ef7-aa57-971548ca8b65 None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:46.411014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 639/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Thu Sep 22 14:48:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:46.417657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e66a9a-9c96-4503-9982-2aa351032022 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:46.461947 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:46.542413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e66a9a-9c96-4503-9982-2aa351032022 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:46.576035 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a3d9f-687e-4883-a29d-433212138e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:46.576637 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a3d9f-687e-4883-a29d-433212138e5b None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:46.576915 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a3d9f-687e-4883-a29d-433212138e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:46.577191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a3d9f-687e-4883-a29d-433212138e5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:46.577881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a3d9f-687e-4883-a29d-433212138e5b None admin] RBAC: Adding query filter params (interface=public, service_id=e9154b768cd24c0d8f8cc22f190239d3, region_id=RegionOne) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:46.578189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a3d9f-687e-4883-a29d-433212138e5b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:46.579577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a3d9f-687e-4883-a29d-433212138e5b None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:46.584418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 661/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Sep 22 14:48:46 2022] GET /identity/v3/endpoints?service_id=e9154b768cd24c0d8f8cc22f190239d3&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:46.590791 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc443184-c94c-4ef7-aa57-971548ca8b65 None admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:46.670457 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:46.688266 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc443184-c94c-4ef7-aa57-971548ca8b65 None admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:46.721796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893cd5b-859c-4dd5-8311-0f0e0b29530a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:46.722387 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893cd5b-859c-4dd5-8311-0f0e0b29530a None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:46.722660 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893cd5b-859c-4dd5-8311-0f0e0b29530a None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:46.722956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893cd5b-859c-4dd5-8311-0f0e0b29530a None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:46.723544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893cd5b-859c-4dd5-8311-0f0e0b29530a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:46.724893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893cd5b-859c-4dd5-8311-0f0e0b29530a None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:46.731675 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 640/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Sep 22 14:48:46 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:48:48.893278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effe07e-bd29-4432-a719-da749fd8aab8 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:48.893810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effe07e-bd29-4432-a719-da749fd8aab8 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:48.893810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effe07e-bd29-4432-a719-da749fd8aab8 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:48.895192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 662/1302] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:48:48.901151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece9091-ea4a-42c0-9cad-520e8bb9d259 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:48.901564 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece9091-ea4a-42c0-9cad-520e8bb9d259 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:48.901697 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece9091-ea4a-42c0-9cad-520e8bb9d259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:48.919578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-cece9091-ea4a-42c0-9cad-520e8bb9d259 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:48:48.920770 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-cece9091-ea4a-42c0-9cad-520e8bb9d259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:48:48.987083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 641/1303] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:48:48 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 14:48:49.255998 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2effe07e-bd29-4432-a719-da749fd8aab8 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:49.353173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:49.360124 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2effe07e-bd29-4432-a719-da749fd8aab8 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:49.392668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f726cc-3d04-4df5-9682-f61bbcdc09d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:49.393289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f726cc-3d04-4df5-9682-f61bbcdc09d0 None admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:49.393573 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f726cc-3d04-4df5-9682-f61bbcdc09d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:49.393843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f726cc-3d04-4df5-9682-f61bbcdc09d0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:49.394420 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f726cc-3d04-4df5-9682-f61bbcdc09d0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:49.395800 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f726cc-3d04-4df5-9682-f61bbcdc09d0 None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:49.405007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application [None req-34f726cc-3d04-4df5-9682-f61bbcdc09d0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:49.405007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:49.405007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 14:48:49.405007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 14:48:49.405007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 14:48:49.405007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 14:48:49.405007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 14:48:49.405007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:48:49.405007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:49.405007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:49.405007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 14:48:49.405007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:49.405007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 14:48:49.405007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 14:48:49.405007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 14:48:49.405007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 14:48:49.405007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 14:48:49.406072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 14:48:49.406072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 14:48:49.406072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 14:48:49.406072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 14:48:49.406072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 14:48:49.406072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 14:48:49.406072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 14:48:49.406072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 14:48:49.406072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 14:48:49.406072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 14:48:49.406072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 14:48:49.406072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 14:48:49.406072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 14:48:49.406072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 14:48:49.406072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 14:48:49.406072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 14:48:49.406977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 14:48:49.406977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application with Lock( Sep 22 14:48:49.406977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 14:48:49.406977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self._enter() Sep 22 14:48:49.406977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 14:48:49.406977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 14:48:49.406977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 14:48:49.406977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application return self.creator() Sep 22 14:48:49.406977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 14:48:49.406977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application created_value = creator( Sep 22 14:48:49.406977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 14:48:49.406977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 14:48:49.406977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 14:48:49.406977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: ERROR keystone.server.flask.application Sep 22 14:48:49.406977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 663/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Sep 22 14:48:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 14:48:49.412457 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece9091-ea4a-42c0-9cad-520e8bb9d259 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:49.488564 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:49.514058 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece9091-ea4a-42c0-9cad-520e8bb9d259 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:49.549442 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6582c-5aa1-4354-a2d9-ee164f22ccfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:49.550045 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6582c-5aa1-4354-a2d9-ee164f22ccfc None admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:49.550321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6582c-5aa1-4354-a2d9-ee164f22ccfc None admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:49.550591 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6582c-5aa1-4354-a2d9-ee164f22ccfc None admin] PATH_INFO: `/v3/services` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:49.551244 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6582c-5aa1-4354-a2d9-ee164f22ccfc None admin] RBAC: Adding query filter params (name=placement) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:48:49.551545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6582c-5aa1-4354-a2d9-ee164f22ccfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:49.552907 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6582c-5aa1-4354-a2d9-ee164f22ccfc None admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:49.558095 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 642/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Thu Sep 22 14:48:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:48:49.563945 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f726cc-3d04-4df5-9682-f61bbcdc09d0 None admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:48:49.663233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f726cc-3d04-4df5-9682-f61bbcdc09d0 None admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:48:49.696215 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939ca1c-5ca1-44b3-8674-874155911eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:48:49.696813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939ca1c-5ca1-44b3-8674-874155911eef None admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:48:49.697089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939ca1c-5ca1-44b3-8674-874155911eef None admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:48:49.697359 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939ca1c-5ca1-44b3-8674-874155911eef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:48:49.698025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939ca1c-5ca1-44b3-8674-874155911eef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:48:49.699403 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939ca1c-5ca1-44b3-8674-874155911eef None admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:48:49.718990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 664/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:48:49 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:48:59.555101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=30 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:48:59.714871 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:49:54.830845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efa0e3-eb9f-4cf5-a881-467040b7515b None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:49:54.831339 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efa0e3-eb9f-4cf5-a881-467040b7515b None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:49:54.831421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efa0e3-eb9f-4cf5-a881-467040b7515b None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:49:54.837343 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 643/1307] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:49:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:49:54.838754 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb44915-1abf-40a1-8ebf-279dfa963a5e None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:49:54.839194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb44915-1abf-40a1-8ebf-279dfa963a5e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:49:54.839362 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb44915-1abf-40a1-8ebf-279dfa963a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:49:54.968109 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-dfb44915-1abf-40a1-8ebf-279dfa963a5e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:49:54.969414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-dfb44915-1abf-40a1-8ebf-279dfa963a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:49:55.090171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 665/1308] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:49:55.387499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76c53c-e60d-4b44-9b18-a9ec10acb7f8 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:49:55.387796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76c53c-e60d-4b44-9b18-a9ec10acb7f8 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:49:55.388042 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76c53c-e60d-4b44-9b18-a9ec10acb7f8 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:49:55.389542 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 644/1309] 10.222.0.26 () {54 vars in 964 bytes} [Thu Sep 22 14:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:49:55.395767 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfe866-c539-45a0-9255-88c1f1373e56 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:49:55.396052 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfe866-c539-45a0-9255-88c1f1373e56 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:49:55.396324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfe866-c539-45a0-9255-88c1f1373e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:49:55.482881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-35cfe866-c539-45a0-9255-88c1f1373e56 None None] MFA Rules not processed for user `f77bbd8de87042aab3674db366635cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:49:55.484132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-35cfe866-c539-45a0-9255-88c1f1373e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:49:55.512643 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:49:55.585151 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 666/1310] 10.222.0.26 () {58 vars in 1061 bytes} [Thu Sep 22 14:49:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:49:55.594392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76c53c-e60d-4b44-9b18-a9ec10acb7f8 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:49:55.596665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-ff76c53c-e60d-4b44-9b18-a9ec10acb7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:49:55.722187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76c53c-e60d-4b44-9b18-a9ec10acb7f8 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:49:55.809298 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe611a18-a773-4bc5-a08c-de4a31330f7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:49:55.810272 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe611a18-a773-4bc5-a08c-de4a31330f7f service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:49:55.810704 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe611a18-a773-4bc5-a08c-de4a31330f7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:49:55.811312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe611a18-a773-4bc5-a08c-de4a31330f7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:49:55.813288 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-fe611a18-a773-4bc5-a08c-de4a31330f7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:49:55.880223 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:49:55.944270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe611a18-a773-4bc5-a08c-de4a31330f7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:49:55.945749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe611a18-a773-4bc5-a08c-de4a31330f7f service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:49:55.968643 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 645/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:49:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:49:58.480873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda75ae8-db91-47d3-92de-0a4ec1821d0f None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:49:58.481398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda75ae8-db91-47d3-92de-0a4ec1821d0f None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:49:58.481398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda75ae8-db91-47d3-92de-0a4ec1821d0f None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:49:58.482816 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 667/1312] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:49:58.488525 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675839d-f59b-4a0e-a6d9-39c4a36dbd7f None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:49:58.489430 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675839d-f59b-4a0e-a6d9-39c4a36dbd7f None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:49:58.489816 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675839d-f59b-4a0e-a6d9-39c4a36dbd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:49:58.561061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-0675839d-f59b-4a0e-a6d9-39c4a36dbd7f None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:49:58.562273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0675839d-f59b-4a0e-a6d9-39c4a36dbd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:49:58.658161 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 646/1313] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:49:58.939092 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda75ae8-db91-47d3-92de-0a4ec1821d0f None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:49:58.940641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-bda75ae8-db91-47d3-92de-0a4ec1821d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:49:59.044362 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda75ae8-db91-47d3-92de-0a4ec1821d0f None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:49:59.116555 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:49:59.118933 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf88bd-3f28-4600-a144-e5efcfe2a329 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:49:59.119726 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf88bd-3f28-4600-a144-e5efcfe2a329 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:49:59.120056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf88bd-3f28-4600-a144-e5efcfe2a329 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:49:59.120521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf88bd-3f28-4600-a144-e5efcfe2a329 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:49:59.122384 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-02bf88bd-3f28-4600-a144-e5efcfe2a329 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:49:59.232836 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf88bd-3f28-4600-a144-e5efcfe2a329 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:49:59.234182 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf88bd-3f28-4600-a144-e5efcfe2a329 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:49:59.252134 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 668/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:49:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:01.723050 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a67bd-938f-4b98-be33-32caf2b788a3 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:01.723562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a67bd-938f-4b98-be33-32caf2b788a3 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:01.723562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a67bd-938f-4b98-be33-32caf2b788a3 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:01.724729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 647/1315] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:01.730419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47a013-0683-4763-910f-a0394c5d5870 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:01.730898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47a013-0683-4763-910f-a0394c5d5870 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:01.731008 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47a013-0683-4763-910f-a0394c5d5870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:01.799025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-cc47a013-0683-4763-910f-a0394c5d5870 None None] MFA Rules not processed for user `b0b665d93f28427a85c1b52614174fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:01.800288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-cc47a013-0683-4763-910f-a0394c5d5870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:01.856468 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:01.903840 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 669/1316] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:01 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:02.194599 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a67bd-938f-4b98-be33-32caf2b788a3 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:02.196152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-387a67bd-938f-4b98-be33-32caf2b788a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:02.205896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:02.300338 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a67bd-938f-4b98-be33-32caf2b788a3 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:02.373949 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2629786a-6f52-45f6-92ee-28616011cbb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:02.374537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629786a-6f52-45f6-92ee-28616011cbb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:02.374836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629786a-6f52-45f6-92ee-28616011cbb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:02.375104 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2629786a-6f52-45f6-92ee-28616011cbb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:02.376670 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2629786a-6f52-45f6-92ee-28616011cbb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:02.437204 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:02.484236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629786a-6f52-45f6-92ee-28616011cbb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:02.485690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2629786a-6f52-45f6-92ee-28616011cbb6 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:02.504432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 648/1317] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:02 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:05.213125 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c8304-54d4-403d-a1ef-b289799332a5 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:05.213664 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c8304-54d4-403d-a1ef-b289799332a5 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:05.213664 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c8304-54d4-403d-a1ef-b289799332a5 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:05.215031 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 670/1318] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:05.220496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcc59d-3ddd-4d85-939b-0afb3c021348 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:05.220887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcc59d-3ddd-4d85-939b-0afb3c021348 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:05.221049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bcc59d-3ddd-4d85-939b-0afb3c021348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:05.286017 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-d3bcc59d-3ddd-4d85-939b-0afb3c021348 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:05.287746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d3bcc59d-3ddd-4d85-939b-0afb3c021348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:05.382796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 649/1319] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:05.676418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c8304-54d4-403d-a1ef-b289799332a5 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:05.678464 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ba7c8304-54d4-403d-a1ef-b289799332a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:05.794953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c8304-54d4-403d-a1ef-b289799332a5 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:05.877438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b3636-0823-410b-8ccb-88b4bb769886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:05.878047 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b3636-0823-410b-8ccb-88b4bb769886 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:05.878324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b3636-0823-410b-8ccb-88b4bb769886 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:05.878592 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b3636-0823-410b-8ccb-88b4bb769886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:05.880221 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3f1b3636-0823-410b-8ccb-88b4bb769886 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:05.888072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:05.994094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b3636-0823-410b-8ccb-88b4bb769886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:05.995477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b3636-0823-410b-8ccb-88b4bb769886 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:06.013724 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 671/1320] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:08.849983 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f07893-3e8a-4d26-8c2f-840b268cdb38 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:08.850521 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f07893-3e8a-4d26-8c2f-840b268cdb38 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:08.850521 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f07893-3e8a-4d26-8c2f-840b268cdb38 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:08.851909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 650/1321] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:50:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:08.857317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dea49-7a1a-40e2-81ac-076acbcf6cf5 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:08.857774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dea49-7a1a-40e2-81ac-076acbcf6cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:08.857905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dea49-7a1a-40e2-81ac-076acbcf6cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:08.932125 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-bd1dea49-7a1a-40e2-81ac-076acbcf6cf5 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:08.933558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-bd1dea49-7a1a-40e2-81ac-076acbcf6cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:08.999373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:09.043077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 672/1322] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:09.331817 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f07893-3e8a-4d26-8c2f-840b268cdb38 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:09.333345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-42f07893-3e8a-4d26-8c2f-840b268cdb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:09.355601 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:09.444471 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f07893-3e8a-4d26-8c2f-840b268cdb38 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:09.519062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a23842-b079-4cc6-9f17-1a393071cad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:09.519672 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a23842-b079-4cc6-9f17-1a393071cad7 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:09.519951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a23842-b079-4cc6-9f17-1a393071cad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:09.520221 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a23842-b079-4cc6-9f17-1a393071cad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:09.522339 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c9a23842-b079-4cc6-9f17-1a393071cad7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:09.594160 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:09.633892 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a23842-b079-4cc6-9f17-1a393071cad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:09.635374 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a23842-b079-4cc6-9f17-1a393071cad7 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:09.653906 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 651/1323] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:50:11.941708 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9143e-7afc-44c3-8c9b-50e1d2df3544 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:11.942231 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9143e-7afc-44c3-8c9b-50e1d2df3544 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:11.942231 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9143e-7afc-44c3-8c9b-50e1d2df3544 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:11.943719 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 673/1324] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:11.949535 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055111d-1b3e-4410-88bd-bad29443a57c None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:11.949949 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055111d-1b3e-4410-88bd-bad29443a57c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:11.950164 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055111d-1b3e-4410-88bd-bad29443a57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:12.019648 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-4055111d-1b3e-4410-88bd-bad29443a57c None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:12.020772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4055111d-1b3e-4410-88bd-bad29443a57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:12.129486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 652/1325] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:12.410161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9143e-7afc-44c3-8c9b-50e1d2df3544 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:12.412352 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-57d9143e-7afc-44c3-8c9b-50e1d2df3544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:12.515967 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9143e-7afc-44c3-8c9b-50e1d2df3544 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:12.593492 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1edcc1c-c84f-4fc6-910e-61987ebd2fef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:12.594018 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1edcc1c-c84f-4fc6-910e-61987ebd2fef service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:12.594296 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1edcc1c-c84f-4fc6-910e-61987ebd2fef service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:12.594853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1edcc1c-c84f-4fc6-910e-61987ebd2fef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:12.596432 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d1edcc1c-c84f-4fc6-910e-61987ebd2fef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:12.614126 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:12.712687 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1edcc1c-c84f-4fc6-910e-61987ebd2fef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:12.714164 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1edcc1c-c84f-4fc6-910e-61987ebd2fef service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:12.732950 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 674/1326] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:14.949586 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cca89-d054-4553-8c4e-40c050ac684f None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:14.950072 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cca89-d054-4553-8c4e-40c050ac684f None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:14.950072 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cca89-d054-4553-8c4e-40c050ac684f None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:14.951502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 653/1327] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:14.957125 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8860f-8d02-4695-9fa8-6d1629f41224 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:14.957557 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8860f-8d02-4695-9fa8-6d1629f41224 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:14.957675 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d8860f-8d02-4695-9fa8-6d1629f41224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:15.028264 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-19d8860f-8d02-4695-9fa8-6d1629f41224 None None] MFA Rules not processed for user `b0b665d93f28427a85c1b52614174fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:15.029488 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-19d8860f-8d02-4695-9fa8-6d1629f41224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:15.095393 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:15.127072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 675/1328] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:14 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:50:15.415380 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cca89-d054-4553-8c4e-40c050ac684f None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:15.416929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-729cca89-d054-4553-8c4e-40c050ac684f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:15.451084 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:15.530087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cca89-d054-4553-8c4e-40c050ac684f None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:15.605690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6ff6f-abc1-471c-81e0-45cdde9e67b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:15.606299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6ff6f-abc1-471c-81e0-45cdde9e67b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:15.606570 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6ff6f-abc1-471c-81e0-45cdde9e67b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:15.606870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6ff6f-abc1-471c-81e0-45cdde9e67b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:15.608440 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-8fe6ff6f-abc1-471c-81e0-45cdde9e67b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:15.685739 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:15.720402 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6ff6f-abc1-471c-81e0-45cdde9e67b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:15.721830 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6ff6f-abc1-471c-81e0-45cdde9e67b6 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:15.739627 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 654/1329] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:15 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:18.119067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436935e-5698-4640-a933-f83b532ef1dd None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:18.119654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436935e-5698-4640-a933-f83b532ef1dd None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:18.119654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436935e-5698-4640-a933-f83b532ef1dd None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:18.120964 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 676/1330] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:18.126420 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba5cd4-af36-463b-8dc8-fa2dc7aac0f0 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:18.126877 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba5cd4-af36-463b-8dc8-fa2dc7aac0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:18.126989 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba5cd4-af36-463b-8dc8-fa2dc7aac0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:18.193248 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-27ba5cd4-af36-463b-8dc8-fa2dc7aac0f0 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:18.194451 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-27ba5cd4-af36-463b-8dc8-fa2dc7aac0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:18.294746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 655/1331] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:18.573334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436935e-5698-4640-a933-f83b532ef1dd None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:18.574922 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-6436935e-5698-4640-a933-f83b532ef1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:18.678441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436935e-5698-4640-a933-f83b532ef1dd None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:18.752478 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2f7f3-4c2b-401d-a1e1-d28cfccaf806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:18.753077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2f7f3-4c2b-401d-a1e1-d28cfccaf806 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:18.753357 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2f7f3-4c2b-401d-a1e1-d28cfccaf806 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:18.753634 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2f7f3-4c2b-401d-a1e1-d28cfccaf806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:18.755785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-68a2f7f3-4c2b-401d-a1e1-d28cfccaf806 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:18.783189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:18.872161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2f7f3-4c2b-401d-a1e1-d28cfccaf806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:18.873516 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2f7f3-4c2b-401d-a1e1-d28cfccaf806 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:18.892893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 677/1332] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:21.121759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa881b-fb2a-4b3b-8b4b-08ee689bbaac None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:21.122331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa881b-fb2a-4b3b-8b4b-08ee689bbaac None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:21.122331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa881b-fb2a-4b3b-8b4b-08ee689bbaac None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:21.123668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 656/1333] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:21.129105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab3612-80cf-4b73-a4bc-63a2f2734d07 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:21.129535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab3612-80cf-4b73-a4bc-63a2f2734d07 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:21.129629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab3612-80cf-4b73-a4bc-63a2f2734d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:21.193690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-d8ab3612-80cf-4b73-a4bc-63a2f2734d07 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:21.194927 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d8ab3612-80cf-4b73-a4bc-63a2f2734d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:21.279907 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:21.305887 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 678/1334] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:21.597470 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa881b-fb2a-4b3b-8b4b-08ee689bbaac None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:21.599681 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-25aa881b-fb2a-4b3b-8b4b-08ee689bbaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:21.642953 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:21.704929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa881b-fb2a-4b3b-8b4b-08ee689bbaac None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:21.780193 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a8d21-fea1-4857-a5ab-629401e0286f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:21.780795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a8d21-fea1-4857-a5ab-629401e0286f service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:21.781067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a8d21-fea1-4857-a5ab-629401e0286f service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:21.781334 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a8d21-fea1-4857-a5ab-629401e0286f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:21.782932 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-830a8d21-fea1-4857-a5ab-629401e0286f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:21.867022 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:21.896186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a8d21-fea1-4857-a5ab-629401e0286f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:21.897666 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a8d21-fea1-4857-a5ab-629401e0286f service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:21.917047 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 657/1335] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:24.409104 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77d68f-ef3e-4ecd-8601-bd8d4d188922 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:24.409628 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77d68f-ef3e-4ecd-8601-bd8d4d188922 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:24.409628 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77d68f-ef3e-4ecd-8601-bd8d4d188922 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:24.411080 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 679/1336] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:24.417102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8642d-091c-4424-a9c9-93866913b532 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:24.417661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8642d-091c-4424-a9c9-93866913b532 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:24.417661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8642d-091c-4424-a9c9-93866913b532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:24.484105 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-99a8642d-091c-4424-a9c9-93866913b532 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:24.485229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-99a8642d-091c-4424-a9c9-93866913b532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:24.587854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 658/1337] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:24.889129 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfae5cc-ed51-4d3c-9d55-5dc6f3c39559 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:24.889411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfae5cc-ed51-4d3c-9d55-5dc6f3c39559 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:24.889688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfae5cc-ed51-4d3c-9d55-5dc6f3c39559 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:24.891083 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 680/1338] 10.222.0.26 () {54 vars in 964 bytes} [Thu Sep 22 14:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:24.901230 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecc205-08e5-4ab5-90ad-748bddc93cf7 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:24.901514 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecc205-08e5-4ab5-90ad-748bddc93cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:24.901786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ecc205-08e5-4ab5-90ad-748bddc93cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:24.958599 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:24.981302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-33ecc205-08e5-4ab5-90ad-748bddc93cf7 None None] MFA Rules not processed for user `f77bbd8de87042aab3674db366635cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:24.982911 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-33ecc205-08e5-4ab5-90ad-748bddc93cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:25.063753 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 659/1339] 10.222.0.26 () {58 vars in 1061 bytes} [Thu Sep 22 14:50:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:25.073702 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfae5cc-ed51-4d3c-9d55-5dc6f3c39559 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:25.075244 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ccfae5cc-ed51-4d3c-9d55-5dc6f3c39559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:25.180419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfae5cc-ed51-4d3c-9d55-5dc6f3c39559 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:25.253782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1e1fd-6dff-42ca-94d7-d5ccd62a7316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:25.254383 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1e1fd-6dff-42ca-94d7-d5ccd62a7316 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:25.254661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1e1fd-6dff-42ca-94d7-d5ccd62a7316 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:25.255501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1e1fd-6dff-42ca-94d7-d5ccd62a7316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:25.257086 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-14e1e1fd-6dff-42ca-94d7-d5ccd62a7316 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:25.292358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:25.368203 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1e1fd-6dff-42ca-94d7-d5ccd62a7316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:25.369675 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1e1fd-6dff-42ca-94d7-d5ccd62a7316 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:25.388667 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 681/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:30.127247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44791c7-f40b-4a7a-a6fa-094043693710 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:30.127247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44791c7-f40b-4a7a-a6fa-094043693710 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:30.127805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44791c7-f40b-4a7a-a6fa-094043693710 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:30.128858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 660/1341] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:30.134557 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce1785-afb7-4557-9bcc-67df617f38be None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:30.134977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce1785-afb7-4557-9bcc-67df617f38be None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:30.135068 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce1785-afb7-4557-9bcc-67df617f38be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:30.206480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-fcce1785-afb7-4557-9bcc-67df617f38be None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:30.207841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-fcce1785-afb7-4557-9bcc-67df617f38be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:30.308552 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:30.321044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 682/1342] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:30.608835 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44791c7-f40b-4a7a-a6fa-094043693710 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:30.610386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f44791c7-f40b-4a7a-a6fa-094043693710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:30.717499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44791c7-f40b-4a7a-a6fa-094043693710 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:30.757163 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:30.793635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f6848-6676-4d96-bfce-7714797cbeec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:30.794347 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6848-6676-4d96-bfce-7714797cbeec service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:30.794630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6848-6676-4d96-bfce-7714797cbeec service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:30.794929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f6848-6676-4d96-bfce-7714797cbeec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:30.796525 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-304f6848-6676-4d96-bfce-7714797cbeec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:30.910320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f6848-6676-4d96-bfce-7714797cbeec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:30.912283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f6848-6676-4d96-bfce-7714797cbeec service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:30.930927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 661/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:33.751527 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a570b-ba22-482e-957b-b947478570ec None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:33.752055 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a570b-ba22-482e-957b-b947478570ec None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:33.752055 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a570b-ba22-482e-957b-b947478570ec None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:33.753310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 683/1344] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:33.758562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7385c99-8c38-4aaa-91ff-4a9b01a41b0b None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:33.759039 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7385c99-8c38-4aaa-91ff-4a9b01a41b0b None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:33.759153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7385c99-8c38-4aaa-91ff-4a9b01a41b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:33.825312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-f7385c99-8c38-4aaa-91ff-4a9b01a41b0b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:33.826511 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f7385c99-8c38-4aaa-91ff-4a9b01a41b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:33.844943 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:33.927857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 662/1345] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:34.226007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a570b-ba22-482e-957b-b947478570ec None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:34.227644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7c4a570b-ba22-482e-957b-b947478570ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:34.343821 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a570b-ba22-482e-957b-b947478570ec None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:34.422100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e65d3-486f-452e-8a9b-fe6b20cd955f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:34.422704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e65d3-486f-452e-8a9b-fe6b20cd955f service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:34.423007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e65d3-486f-452e-8a9b-fe6b20cd955f service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:34.423269 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e65d3-486f-452e-8a9b-fe6b20cd955f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:34.424837 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-316e65d3-486f-452e-8a9b-fe6b20cd955f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:34.470089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:34.541173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e65d3-486f-452e-8a9b-fe6b20cd955f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:34.542610 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e65d3-486f-452e-8a9b-fe6b20cd955f service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:34.562421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 684/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:38.777539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd2ef5-9149-423f-8b84-ad5fc576030c None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:38.778074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd2ef5-9149-423f-8b84-ad5fc576030c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:38.778074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd2ef5-9149-423f-8b84-ad5fc576030c None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:38.779849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 663/1347] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:38.785324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42e50c-ec67-4231-9167-96d4de744d67 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:38.785735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42e50c-ec67-4231-9167-96d4de744d67 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:38.785981 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42e50c-ec67-4231-9167-96d4de744d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:38.858634 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-ea42e50c-ec67-4231-9167-96d4de744d67 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:38.859893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ea42e50c-ec67-4231-9167-96d4de744d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:38.956349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:38.959848 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 685/1348] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:39.245963 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd2ef5-9149-423f-8b84-ad5fc576030c None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:39.247559 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-33fd2ef5-9149-423f-8b84-ad5fc576030c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:39.356835 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd2ef5-9149-423f-8b84-ad5fc576030c None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:39.405383 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:39.432276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec9b50-909c-4377-8808-469957d474ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:39.432886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec9b50-909c-4377-8808-469957d474ab service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:39.433169 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec9b50-909c-4377-8808-469957d474ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:39.433459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec9b50-909c-4377-8808-469957d474ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:39.435073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4aec9b50-909c-4377-8808-469957d474ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:39.545942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec9b50-909c-4377-8808-469957d474ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:39.547471 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec9b50-909c-4377-8808-469957d474ab service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:39.566444 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 664/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:50:44.588469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504793d-1294-4773-95c5-a6a57e921fa8 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:44.588942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504793d-1294-4773-95c5-a6a57e921fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:44.589061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504793d-1294-4773-95c5-a6a57e921fa8 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:44.590410 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 686/1350] 10.222.0.26 () {54 vars in 921 bytes} [Thu Sep 22 14:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:44.595865 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48af14-a1d4-415c-bea2-be2d832c6771 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:44.596616 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48af14-a1d4-415c-bea2-be2d832c6771 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:44.597059 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48af14-a1d4-415c-bea2-be2d832c6771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:44.711383 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-be48af14-a1d4-415c-bea2-be2d832c6771 None None] MFA Rules not processed for user `47b6b34b4ee5492fb1f0699716e5118b`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:44.713508 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-be48af14-a1d4-415c-bea2-be2d832c6771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:44.739349 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:44.888199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 665/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Sep 22 14:50:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:49.176824 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a72d60-47eb-4ccb-9fbb-a5f253b7347e None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:49.177441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a72d60-47eb-4ccb-9fbb-a5f253b7347e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:49.177441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a72d60-47eb-4ccb-9fbb-a5f253b7347e None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:49.178798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 687/1352] 10.222.0.26 () {54 vars in 921 bytes} [Thu Sep 22 14:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:49.184292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45408b4-9a08-43e3-9789-00512e4f45fa None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:49.184805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45408b4-9a08-43e3-9789-00512e4f45fa None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:49.184910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45408b4-9a08-43e3-9789-00512e4f45fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:49.283379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-a45408b4-9a08-43e3-9789-00512e4f45fa None None] MFA Rules not processed for user `47b6b34b4ee5492fb1f0699716e5118b`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:49.284561 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a45408b4-9a08-43e3-9789-00512e4f45fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:49.378152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:49.424704 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 666/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Sep 22 14:50:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:50.832075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8765e7-12e2-4781-884c-577c2c10290f None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:50.832632 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8765e7-12e2-4781-884c-577c2c10290f None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:50.832728 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8765e7-12e2-4781-884c-577c2c10290f None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:50.834121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 688/1354] 10.222.0.26 () {54 vars in 921 bytes} [Thu Sep 22 14:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:50.839342 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e37ea-8d6e-46bd-9b8e-c86575e09177 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:50.839820 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e37ea-8d6e-46bd-9b8e-c86575e09177 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:50.840019 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e37ea-8d6e-46bd-9b8e-c86575e09177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:50.919666 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-b74e37ea-8d6e-46bd-9b8e-c86575e09177 None None] MFA Rules not processed for user `47b6b34b4ee5492fb1f0699716e5118b`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:50.920872 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b74e37ea-8d6e-46bd-9b8e-c86575e09177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:51.040186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 667/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Sep 22 14:50:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:54.528635 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed4379-8736-4bec-b5a5-32b4216d4a64 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:54.529634 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed4379-8736-4bec-b5a5-32b4216d4a64 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:54.530452 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eed4379-8736-4bec-b5a5-32b4216d4a64 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:54.536752 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 689/1356] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:50:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:54.538000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5a6d2-a754-4f14-b5b9-4608c87c96c0 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:54.538423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5a6d2-a754-4f14-b5b9-4608c87c96c0 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:54.538620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5a6d2-a754-4f14-b5b9-4608c87c96c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:54.579170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:54.620918 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-a0b5a6d2-a754-4f14-b5b9-4608c87c96c0 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:54.622233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a0b5a6d2-a754-4f14-b5b9-4608c87c96c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:54.733551 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 668/1357] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:55.036287 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be67d84-3448-41ef-ac5a-904972392b00 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:55.036948 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be67d84-3448-41ef-ac5a-904972392b00 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:55.037415 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be67d84-3448-41ef-ac5a-904972392b00 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:55.039819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 690/1358] 10.222.0.26 () {54 vars in 961 bytes} [Thu Sep 22 14:50:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:55.048156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b1637-d48e-4f03-b74d-bf46e371c94f None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:55.048523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b1637-d48e-4f03-b74d-bf46e371c94f None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:55.048888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b1637-d48e-4f03-b74d-bf46e371c94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:55.137507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-0d8b1637-d48e-4f03-b74d-bf46e371c94f None None] MFA Rules not processed for user `0c9be41c39494e54a14398a1f03c4d69`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:55.138710 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0d8b1637-d48e-4f03-b74d-bf46e371c94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:55.191874 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:55.269458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 669/1359] 10.222.0.26 () {58 vars in 1058 bytes} [Thu Sep 22 14:50:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:55.277498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be67d84-3448-41ef-ac5a-904972392b00 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:55.279140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7be67d84-3448-41ef-ac5a-904972392b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:55.449146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be67d84-3448-41ef-ac5a-904972392b00 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:55.525589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:55.568342 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e9e68-52aa-47bb-b721-9cd6a9416a2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:55.569419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e9e68-52aa-47bb-b721-9cd6a9416a2f service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:55.569419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e9e68-52aa-47bb-b721-9cd6a9416a2f service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:55.569676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e9e68-52aa-47bb-b721-9cd6a9416a2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:55.571083 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-2d6e9e68-52aa-47bb-b721-9cd6a9416a2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:55.669685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e9e68-52aa-47bb-b721-9cd6a9416a2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:55.671376 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e9e68-52aa-47bb-b721-9cd6a9416a2f service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:55.693188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 691/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:50:58.146157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739b00b-8b66-4a78-983d-65d048d6a231 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:58.146633 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739b00b-8b66-4a78-983d-65d048d6a231 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:58.146633 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739b00b-8b66-4a78-983d-65d048d6a231 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:58.150492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 670/1361] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:50:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:50:58.153578 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b2395-569e-41c2-9e76-f12ded13e3aa None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:58.154039 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b2395-569e-41c2-9e76-f12ded13e3aa None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:58.154039 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b2395-569e-41c2-9e76-f12ded13e3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:58.223151 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-f31b2395-569e-41c2-9e76-f12ded13e3aa None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:50:58.224364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-f31b2395-569e-41c2-9e76-f12ded13e3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:58.265522 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:58.334367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 692/1362] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:50:58.623840 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739b00b-8b66-4a78-983d-65d048d6a231 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:50:58.625334 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4739b00b-8b66-4a78-983d-65d048d6a231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:58.804890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739b00b-8b66-4a78-983d-65d048d6a231 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:50:58.818389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:50:58.941546 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faadf41a-94c4-4278-ab2a-dd8605d7b448 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:50:58.942177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadf41a-94c4-4278-ab2a-dd8605d7b448 service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:50:58.942453 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadf41a-94c4-4278-ab2a-dd8605d7b448 service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:50:58.942716 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadf41a-94c4-4278-ab2a-dd8605d7b448 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:50:58.944311 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-faadf41a-94c4-4278-ab2a-dd8605d7b448 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:50:59.049095 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faadf41a-94c4-4278-ab2a-dd8605d7b448 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:50:59.050706 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faadf41a-94c4-4278-ab2a-dd8605d7b448 service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:50:59.069501 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 671/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:50:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:01.591150 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cffa27-f28d-4b73-89b2-b6208aa76c13 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:01.591751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cffa27-f28d-4b73-89b2-b6208aa76c13 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:01.591751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cffa27-f28d-4b73-89b2-b6208aa76c13 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:01.593250 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 693/1364] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:01.599292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a4c48-99c1-4410-91ec-9a2c39a682ce None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:01.599755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a4c48-99c1-4410-91ec-9a2c39a682ce None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:01.600010 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a4c48-99c1-4410-91ec-9a2c39a682ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:01.615971 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:01.670280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-554a4c48-99c1-4410-91ec-9a2c39a682ce None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:01.671539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-554a4c48-99c1-4410-91ec-9a2c39a682ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:01.776897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 672/1365] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:02.070306 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339f9ca-5b14-48da-994c-8311b3d376aa None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:02.070862 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339f9ca-5b14-48da-994c-8311b3d376aa None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:02.071282 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339f9ca-5b14-48da-994c-8311b3d376aa None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:02.073179 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 694/1366] 10.222.0.26 () {54 vars in 961 bytes} [Thu Sep 22 14:51:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:02.078938 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d642ce-7471-4265-bb96-40ef1eff9623 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:02.079352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d642ce-7471-4265-bb96-40ef1eff9623 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:02.079788 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d642ce-7471-4265-bb96-40ef1eff9623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:02.185847 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-48d642ce-7471-4265-bb96-40ef1eff9623 None None] MFA Rules not processed for user `0c9be41c39494e54a14398a1f03c4d69`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:02.187407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-48d642ce-7471-4265-bb96-40ef1eff9623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:02.229612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:02.321745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 673/1367] 10.222.0.26 () {58 vars in 1058 bytes} [Thu Sep 22 14:51:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:02.331109 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339f9ca-5b14-48da-994c-8311b3d376aa None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:02.333278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-9339f9ca-5b14-48da-994c-8311b3d376aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:02.498943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339f9ca-5b14-48da-994c-8311b3d376aa None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:02.523774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:02.629390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c66cf7-7abe-4fc7-b451-1ba8ef47605f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:02.630060 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c66cf7-7abe-4fc7-b451-1ba8ef47605f service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:02.630331 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c66cf7-7abe-4fc7-b451-1ba8ef47605f service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:02.630438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c66cf7-7abe-4fc7-b451-1ba8ef47605f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:02.632101 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-96c66cf7-7abe-4fc7-b451-1ba8ef47605f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:02.744122 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c66cf7-7abe-4fc7-b451-1ba8ef47605f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:02.745738 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c66cf7-7abe-4fc7-b451-1ba8ef47605f service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:02.765247 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 695/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:02.813628 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc10a5-042d-42d7-911e-414c8544359c None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:02.814131 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc10a5-042d-42d7-911e-414c8544359c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:02.814226 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc10a5-042d-42d7-911e-414c8544359c None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:02.815702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 674/1369] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:02.821573 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0223e7c-1772-44e5-b544-7ca47f449f56 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:02.822620 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0223e7c-1772-44e5-b544-7ca47f449f56 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:02.823150 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0223e7c-1772-44e5-b544-7ca47f449f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:02.860243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:02.911648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-e0223e7c-1772-44e5-b544-7ca47f449f56 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:02.912954 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e0223e7c-1772-44e5-b544-7ca47f449f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:03.050776 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 696/1370] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:03.384367 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f52fd5-9f6d-4e72-9ed3-2258d28496e9 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:03.384779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f52fd5-9f6d-4e72-9ed3-2258d28496e9 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:03.385052 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f52fd5-9f6d-4e72-9ed3-2258d28496e9 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:03.386596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 675/1371] 10.222.0.26 () {54 vars in 963 bytes} [Thu Sep 22 14:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:03.391469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408e2c3-a812-48d6-afcb-66401c09838c None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:03.391837 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408e2c3-a812-48d6-afcb-66401c09838c None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:03.392174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408e2c3-a812-48d6-afcb-66401c09838c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:03.476990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-c408e2c3-a812-48d6-afcb-66401c09838c None None] MFA Rules not processed for user `1d265d111b534944b7812cf067e317a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:03.479564 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-c408e2c3-a812-48d6-afcb-66401c09838c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:03.535964 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:03.573459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 697/1372] 10.222.0.26 () {58 vars in 1060 bytes} [Thu Sep 22 14:51:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:51:03.580814 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f52fd5-9f6d-4e72-9ed3-2258d28496e9 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:03.582971 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-78f52fd5-9f6d-4e72-9ed3-2258d28496e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:03.692391 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f52fd5-9f6d-4e72-9ed3-2258d28496e9 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:03.737462 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:03.778656 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63ecf2-4aa1-41d6-949d-ff2fbe1ce8e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:03.779314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63ecf2-4aa1-41d6-949d-ff2fbe1ce8e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:03.779590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63ecf2-4aa1-41d6-949d-ff2fbe1ce8e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:03.779858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63ecf2-4aa1-41d6-949d-ff2fbe1ce8e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:03.781763 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-aa63ecf2-4aa1-41d6-949d-ff2fbe1ce8e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:03.917275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63ecf2-4aa1-41d6-949d-ff2fbe1ce8e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:03.918745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63ecf2-4aa1-41d6-949d-ff2fbe1ce8e6 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:03.939579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 676/1373] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:05.010070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7756c07-2599-41b0-a2e1-1c5133780172 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:05.010656 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7756c07-2599-41b0-a2e1-1c5133780172 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:05.010656 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7756c07-2599-41b0-a2e1-1c5133780172 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:05.012164 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 698/1374] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:05.017729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25354729-a9d8-4b31-a84a-b03264acaebd None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:05.018156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25354729-a9d8-4b31-a84a-b03264acaebd None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:05.018280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25354729-a9d8-4b31-a84a-b03264acaebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:05.085403 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-25354729-a9d8-4b31-a84a-b03264acaebd None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:05.086626 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-25354729-a9d8-4b31-a84a-b03264acaebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:05.108103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:05.185965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 677/1375] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:05.478404 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7756c07-2599-41b0-a2e1-1c5133780172 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:05.479983 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d7756c07-2599-41b0-a2e1-1c5133780172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:05.645080 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7756c07-2599-41b0-a2e1-1c5133780172 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:05.698647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:05.763441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b10106-9c16-4067-a386-92dcebb51b5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:05.764062 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b10106-9c16-4067-a386-92dcebb51b5a service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:05.764337 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b10106-9c16-4067-a386-92dcebb51b5a service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:05.764605 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b10106-9c16-4067-a386-92dcebb51b5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:05.766204 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-19b10106-9c16-4067-a386-92dcebb51b5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:05.896621 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b10106-9c16-4067-a386-92dcebb51b5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:05.898165 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b10106-9c16-4067-a386-92dcebb51b5a service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:05.917474 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 699/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:08.198285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba5013-c8ab-4b44-a818-6a8002a2f3a3 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:08.198966 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba5013-c8ab-4b44-a818-6a8002a2f3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:08.198966 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba5013-c8ab-4b44-a818-6a8002a2f3a3 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:08.200346 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 678/1377] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:08.205577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf52231-40fd-41c1-a0bf-6c03a3fe3ea2 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:08.206021 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf52231-40fd-41c1-a0bf-6c03a3fe3ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:08.206252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf52231-40fd-41c1-a0bf-6c03a3fe3ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:08.274692 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-fcf52231-40fd-41c1-a0bf-6c03a3fe3ea2 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:08.275903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-fcf52231-40fd-41c1-a0bf-6c03a3fe3ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:08.282942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:08.401298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 700/1378] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:08.694380 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba5013-c8ab-4b44-a818-6a8002a2f3a3 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:08.696060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2cba5013-c8ab-4b44-a818-6a8002a2f3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:08.856068 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba5013-c8ab-4b44-a818-6a8002a2f3a3 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:08.860132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:08.985039 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36d9d8-1b3e-4181-b713-c874f08dfcab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:08.985549 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d9d8-1b3e-4181-b713-c874f08dfcab service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:08.985828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d9d8-1b3e-4181-b713-c874f08dfcab service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:08.986099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d9d8-1b3e-4181-b713-c874f08dfcab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:08.988278 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-da36d9d8-1b3e-4181-b713-c874f08dfcab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:09.092510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36d9d8-1b3e-4181-b713-c874f08dfcab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:09.094137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36d9d8-1b3e-4181-b713-c874f08dfcab service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:09.114648 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 679/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:11.368214 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191278e5-dd59-4d0a-972f-a22f6d5e2597 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:11.368670 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191278e5-dd59-4d0a-972f-a22f6d5e2597 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:11.368770 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191278e5-dd59-4d0a-972f-a22f6d5e2597 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:11.370098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 701/1380] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:11.375598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec463077-1de1-4977-9404-7b625d4b072b None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:11.376033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec463077-1de1-4977-9404-7b625d4b072b None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:11.376199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec463077-1de1-4977-9404-7b625d4b072b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:11.387787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:11.467773 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-ec463077-1de1-4977-9404-7b625d4b072b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:11.469021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-ec463077-1de1-4977-9404-7b625d4b072b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:11.595543 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 680/1381] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:11.946226 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191278e5-dd59-4d0a-972f-a22f6d5e2597 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:11.947841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-191278e5-dd59-4d0a-972f-a22f6d5e2597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:12.088549 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:12.100452 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191278e5-dd59-4d0a-972f-a22f6d5e2597 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:12.230339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b975ce-f9b6-410a-90e0-a3770e0d96a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:12.230974 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b975ce-f9b6-410a-90e0-a3770e0d96a0 service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:12.231255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b975ce-f9b6-410a-90e0-a3770e0d96a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:12.231529 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b975ce-f9b6-410a-90e0-a3770e0d96a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:12.233106 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-f2b975ce-f9b6-410a-90e0-a3770e0d96a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:12.326292 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b975ce-f9b6-410a-90e0-a3770e0d96a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:12.327957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b975ce-f9b6-410a-90e0-a3770e0d96a0 service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:12.339023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:12.348625 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 702/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:14.583079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956d475-7439-4ee2-938a-aaf464cffa13 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:14.583619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956d475-7439-4ee2-938a-aaf464cffa13 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:14.583619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956d475-7439-4ee2-938a-aaf464cffa13 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:14.585058 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 681/1383] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:14.590368 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa048e-1321-4b2d-8e82-706e3fb36fc5 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:14.590809 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa048e-1321-4b2d-8e82-706e3fb36fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:14.590948 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa048e-1321-4b2d-8e82-706e3fb36fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:14.677204 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-e3aa048e-1321-4b2d-8e82-706e3fb36fc5 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:14.678438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e3aa048e-1321-4b2d-8e82-706e3fb36fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:14.778841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 703/1384] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:51:15.069990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956d475-7439-4ee2-938a-aaf464cffa13 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:15.071669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e956d475-7439-4ee2-938a-aaf464cffa13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:15.175220 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:15.239776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956d475-7439-4ee2-938a-aaf464cffa13 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:15.364643 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e9482-e606-4fa0-8927-4e87f7441d8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:15.365263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e9482-e606-4fa0-8927-4e87f7441d8d service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:15.365539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e9482-e606-4fa0-8927-4e87f7441d8d service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:15.365810 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e9482-e606-4fa0-8927-4e87f7441d8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:15.367450 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-be1e9482-e606-4fa0-8927-4e87f7441d8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:15.421865 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:15.465356 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e9482-e606-4fa0-8927-4e87f7441d8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:15.466980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e9482-e606-4fa0-8927-4e87f7441d8d service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:15.486131 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 682/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:17.717364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37586bb-cf30-4389-ae1d-3dc9d4669be6 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:17.717929 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37586bb-cf30-4389-ae1d-3dc9d4669be6 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:17.717929 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37586bb-cf30-4389-ae1d-3dc9d4669be6 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:17.719300 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 704/1386] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:17.725520 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906c59e-bf8f-482b-a592-d393453a9ac3 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:17.725980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906c59e-bf8f-482b-a592-d393453a9ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:17.726132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906c59e-bf8f-482b-a592-d393453a9ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:17.795622 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-0906c59e-bf8f-482b-a592-d393453a9ac3 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:17.796808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0906c59e-bf8f-482b-a592-d393453a9ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:17.895034 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 683/1387] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:18.184110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37586bb-cf30-4389-ae1d-3dc9d4669be6 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:18.185666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-f37586bb-cf30-4389-ae1d-3dc9d4669be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:18.263746 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:18.343795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37586bb-cf30-4389-ae1d-3dc9d4669be6 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:18.459901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a8223-b8e2-4589-b530-38cfd36276b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:18.460522 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a8223-b8e2-4589-b530-38cfd36276b9 service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:18.460797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a8223-b8e2-4589-b530-38cfd36276b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:18.461219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a8223-b8e2-4589-b530-38cfd36276b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:18.462856 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-744a8223-b8e2-4589-b530-38cfd36276b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:18.506177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:18.560403 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a8223-b8e2-4589-b530-38cfd36276b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:18.562084 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a8223-b8e2-4589-b530-38cfd36276b9 service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:18.581670 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 705/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:19.479757 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568bfe09-be03-43c4-824b-6bd3c230574f None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:19.480291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568bfe09-be03-43c4-824b-6bd3c230574f None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:19.480291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568bfe09-be03-43c4-824b-6bd3c230574f None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:19.481811 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 684/1389] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:19.487762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de69f3-9973-41aa-912a-101e336f860c None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:19.488265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de69f3-9973-41aa-912a-101e336f860c None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:19.488363 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de69f3-9973-41aa-912a-101e336f860c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:19.559101 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-88de69f3-9973-41aa-912a-101e336f860c None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:19.560459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-88de69f3-9973-41aa-912a-101e336f860c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:19.657885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 706/1390] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:19.958859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1f57d-29d1-4340-a26a-d09694e56962 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:19.959148 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1f57d-29d1-4340-a26a-d09694e56962 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:19.959426 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1f57d-29d1-4340-a26a-d09694e56962 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:19.960755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 685/1391] 10.222.0.26 () {54 vars in 962 bytes} [Thu Sep 22 14:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:19.966196 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28aca6-6e18-442b-a640-7c07d02ab98f None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:19.966479 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28aca6-6e18-442b-a640-7c07d02ab98f None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:19.966778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28aca6-6e18-442b-a640-7c07d02ab98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:19.983053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:20.048095 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-6c28aca6-6e18-442b-a640-7c07d02ab98f None None] MFA Rules not processed for user `bb5248cb1c39412680cce0d9891fe1ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:20.049829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-6c28aca6-6e18-442b-a640-7c07d02ab98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:20.125464 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 707/1392] 10.222.0.26 () {58 vars in 1059 bytes} [Thu Sep 22 14:51:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:20.135419 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1f57d-29d1-4340-a26a-d09694e56962 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:20.136956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-90c1f57d-29d1-4340-a26a-d09694e56962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:20.168584 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:20.248004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1f57d-29d1-4340-a26a-d09694e56962 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:20.336693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfebf4-ff00-4278-a7a8-40e65d7e570d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5248cb1c39412680cce0d9891fe1ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:20.337580 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfebf4-ff00-4278-a7a8-40e65d7e570d service glance] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:20.337990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfebf4-ff00-4278-a7a8-40e65d7e570d service glance] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:20.338391 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfebf4-ff00-4278-a7a8-40e65d7e570d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:20.340969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-57dfebf4-ff00-4278-a7a8-40e65d7e570d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:20.429951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:20.463888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dfebf4-ff00-4278-a7a8-40e65d7e570d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:20.465353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dfebf4-ff00-4278-a7a8-40e65d7e570d service glance] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:20.488532 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 686/1393] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:20.791473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f73180-adae-48fa-90e9-68dd3f49527a None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:20.792045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f73180-adae-48fa-90e9-68dd3f49527a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:20.792045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f73180-adae-48fa-90e9-68dd3f49527a None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:20.793672 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 708/1394] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:20.799199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef158ea-356a-444c-8a57-561af8bf1206 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:20.799489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef158ea-356a-444c-8a57-561af8bf1206 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:20.799773 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef158ea-356a-444c-8a57-561af8bf1206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:20.883809 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-7ef158ea-356a-444c-8a57-561af8bf1206 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:20.885328 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7ef158ea-356a-444c-8a57-561af8bf1206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:20.988780 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 687/1395] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:21.241645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e0d72-0cde-4866-90df-eba390c6c64e None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:21.241933 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e0d72-0cde-4866-90df-eba390c6c64e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:21.242209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e0d72-0cde-4866-90df-eba390c6c64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:21.289246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef158ea-356a-444c-8a57-561af8bf1206 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:21.290554 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7ef158ea-356a-444c-8a57-561af8bf1206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:21.317234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-513e0d72-0cde-4866-90df-eba390c6c64e None None] MFA Rules not processed for user `6256407d9df648d5bbe33495cb9a8b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:21.318474 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-513e0d72-0cde-4866-90df-eba390c6c64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:21.335782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:21.381135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:21.415634 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 709/1396] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Sep 22 14:51:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:21.419175 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef158ea-356a-444c-8a57-561af8bf1206 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:21.461801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f863589-fb33-4a7d-9852-1563403bfdca None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:21.462080 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f863589-fb33-4a7d-9852-1563403bfdca None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:21.462353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f863589-fb33-4a7d-9852-1563403bfdca None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:21.463912 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 710/1397] 10.222.0.26 () {54 vars in 943 bytes} [Thu Sep 22 14:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 14:51:21.470267 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc81f70-770d-404b-b54b-02076db054d4 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:21.470548 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc81f70-770d-404b-b54b-02076db054d4 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:21.470861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc81f70-770d-404b-b54b-02076db054d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:21.545019 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb5d1f-22bc-43d4-a196-e4f03c6d3b1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:21.545593 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb5d1f-22bc-43d4-a196-e4f03c6d3b1f service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:21.545875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb5d1f-22bc-43d4-a196-e4f03c6d3b1f service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:21.546135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb5d1f-22bc-43d4-a196-e4f03c6d3b1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:21.547765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-09eb5d1f-22bc-43d4-a196-e4f03c6d3b1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:21.554719 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-2cc81f70-770d-404b-b54b-02076db054d4 None None] MFA Rules not processed for user `0a265623ae734936a340c97df852e227`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:21.555949 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-2cc81f70-770d-404b-b54b-02076db054d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:21.586366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:21.644206 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb5d1f-22bc-43d4-a196-e4f03c6d3b1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:21.645835 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb5d1f-22bc-43d4-a196-e4f03c6d3b1f service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:21.664569 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 688/1398] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:21.672426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 711/1399] 10.222.0.26 () {58 vars in 1040 bytes} [Thu Sep 22 14:51:21 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:51:21.684042 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb5d1f-22bc-43d4-a196-e4f03c6d3b1f service nova] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:21.685263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-09eb5d1f-22bc-43d4-a196-e4f03c6d3b1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:21.789139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb5d1f-22bc-43d4-a196-e4f03c6d3b1f service nova] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:21.871729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:21.918971 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ab5f8-10b6-4d46-8ee7-248e12512aaa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a265623ae734936a340c97df852e227', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:21.919950 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ab5f8-10b6-4d46-8ee7-248e12512aaa service swift] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:21.919950 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ab5f8-10b6-4d46-8ee7-248e12512aaa service swift] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:21.920155 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ab5f8-10b6-4d46-8ee7-248e12512aaa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:21.921749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-859ab5f8-10b6-4d46-8ee7-248e12512aaa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:22.008164 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ab5f8-10b6-4d46-8ee7-248e12512aaa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:22.009765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ab5f8-10b6-4d46-8ee7-248e12512aaa service swift] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:22.019494 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 689/1400] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Sep 22 14:51:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 14:51:23.922587 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bddaee-b4b8-4b0a-ab58-3c640f7db5bd None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:23.923199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bddaee-b4b8-4b0a-ab58-3c640f7db5bd None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:23.923199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bddaee-b4b8-4b0a-ab58-3c640f7db5bd None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:23.924702 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 712/1401] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:23.930140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c4d76-b693-46cc-84a9-3d508e0de6af None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:23.930623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c4d76-b693-46cc-84a9-3d508e0de6af None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:23.930721 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c4d76-b693-46cc-84a9-3d508e0de6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:24.014890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-0d5c4d76-b693-46cc-84a9-3d508e0de6af None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:24.016123 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0d5c4d76-b693-46cc-84a9-3d508e0de6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:24.094588 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:24.145819 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 690/1402] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:24.440720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bddaee-b4b8-4b0a-ab58-3c640f7db5bd None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:24.442248 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-36bddaee-b4b8-4b0a-ab58-3c640f7db5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:24.482749 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:24.614552 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bddaee-b4b8-4b0a-ab58-3c640f7db5bd None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:24.731049 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:24.742336 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba6fcc-a93e-47e3-9ba3-3b601361be94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:24.742909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba6fcc-a93e-47e3-9ba3-3b601361be94 service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:24.743201 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba6fcc-a93e-47e3-9ba3-3b601361be94 service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:24.743447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba6fcc-a93e-47e3-9ba3-3b601361be94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:24.745012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7eba6fcc-a93e-47e3-9ba3-3b601361be94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:24.845679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba6fcc-a93e-47e3-9ba3-3b601361be94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:24.847880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba6fcc-a93e-47e3-9ba3-3b601361be94 service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:24.887065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 713/1403] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:27.210852 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad253a-5277-4374-b2c4-bb304ce5a60b None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:27.211463 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad253a-5277-4374-b2c4-bb304ce5a60b None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:27.211463 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad253a-5277-4374-b2c4-bb304ce5a60b None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:27.214786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 691/1404] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:27.218582 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1d2d7-c292-4b51-be41-8117195e895f None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:27.219729 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1d2d7-c292-4b51-be41-8117195e895f None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:27.220063 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1d2d7-c292-4b51-be41-8117195e895f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:27.297297 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-a0b1d2d7-c292-4b51-be41-8117195e895f None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:27.298520 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a0b1d2d7-c292-4b51-be41-8117195e895f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:27.370415 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:27.412892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 714/1405] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:27.704972 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad253a-5277-4374-b2c4-bb304ce5a60b None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:27.707432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c5ad253a-5277-4374-b2c4-bb304ce5a60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:27.868341 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad253a-5277-4374-b2c4-bb304ce5a60b None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:27.915859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:27.999378 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c4917-0e84-4e44-8d71-a3136af757ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:27.999378 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c4917-0e84-4e44-8d71-a3136af757ad service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:27.999696 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c4917-0e84-4e44-8d71-a3136af757ad service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:27.999917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c4917-0e84-4e44-8d71-a3136af757ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:28.001661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-6d9c4917-0e84-4e44-8d71-a3136af757ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:28.103072 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c4917-0e84-4e44-8d71-a3136af757ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:28.104671 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c4917-0e84-4e44-8d71-a3136af757ad service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:28.123706 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 692/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:30.444189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bd5d8-18a9-4fda-a098-24918361b714 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:30.444751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bd5d8-18a9-4fda-a098-24918361b714 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:30.444751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bd5d8-18a9-4fda-a098-24918361b714 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:30.446236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 715/1407] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:30.451732 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fd930-7c89-40d7-84e1-929179e4cc01 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:30.452271 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fd930-7c89-40d7-84e1-929179e4cc01 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:30.452271 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fd930-7c89-40d7-84e1-929179e4cc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:30.510850 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:30.526064 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-f32fd930-7c89-40d7-84e1-929179e4cc01 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:30.527425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f32fd930-7c89-40d7-84e1-929179e4cc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:30.650348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 693/1408] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:30.948830 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bd5d8-18a9-4fda-a098-24918361b714 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:30.950409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b68bd5d8-18a9-4fda-a098-24918361b714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:31.120251 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bd5d8-18a9-4fda-a098-24918361b714 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:31.175506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:31.253374 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af5b84-bc88-4d80-b161-0ffdd89ea3b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:31.253993 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af5b84-bc88-4d80-b161-0ffdd89ea3b2 service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:31.254277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af5b84-bc88-4d80-b161-0ffdd89ea3b2 service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:31.254550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af5b84-bc88-4d80-b161-0ffdd89ea3b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:31.256194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-74af5b84-bc88-4d80-b161-0ffdd89ea3b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:31.371581 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af5b84-bc88-4d80-b161-0ffdd89ea3b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:31.373308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af5b84-bc88-4d80-b161-0ffdd89ea3b2 service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:31.393769 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 716/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:40.648915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=46 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:41.391310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=76 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:45.307470 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d52561-bc4d-4c0b-b20f-3610144daa52 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:45.308035 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d52561-bc4d-4c0b-b20f-3610144daa52 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:45.308035 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d52561-bc4d-4c0b-b20f-3610144daa52 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:45.309428 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 694/1410] 10.222.0.26 () {54 vars in 918 bytes} [Thu Sep 22 14:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:45.314797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdcea5-f52c-4df7-ad04-a69f9c9660f8 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:45.315249 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdcea5-f52c-4df7-ad04-a69f9c9660f8 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:45.315449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdcea5-f52c-4df7-ad04-a69f9c9660f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:45.391131 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-ddcdcea5-f52c-4df7-ad04-a69f9c9660f8 None None] MFA Rules not processed for user `47b6b34b4ee5492fb1f0699716e5118b`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:45.392355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ddcdcea5-f52c-4df7-ad04-a69f9c9660f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:45.499307 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 717/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Thu Sep 22 14:51:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:45.514229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b95359-11b3-49d0-a36d-98531953ed32 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:45.514505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b95359-11b3-49d0-a36d-98531953ed32 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:45.514800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b95359-11b3-49d0-a36d-98531953ed32 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:45.516140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 695/1412] 10.222.0.26 () {54 vars in 918 bytes} [Thu Sep 22 14:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:45.521809 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec77de-f6dc-4770-a5b9-7781aeb740c4 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:45.522112 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec77de-f6dc-4770-a5b9-7781aeb740c4 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:45.522379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec77de-f6dc-4770-a5b9-7781aeb740c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:45.596133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:45.599485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-fdec77de-f6dc-4770-a5b9-7781aeb740c4 None None] MFA Rules not processed for user `0c9be41c39494e54a14398a1f03c4d69`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:45.601258 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-fdec77de-f6dc-4770-a5b9-7781aeb740c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:45.718935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 718/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Thu Sep 22 14:51:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:48.318615 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a80db-a353-41cb-8ef1-719f059c6a0e None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:48.319195 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a80db-a353-41cb-8ef1-719f059c6a0e None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:48.319195 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a80db-a353-41cb-8ef1-719f059c6a0e None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:48.320515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 696/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:48.326018 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956b5f2-be12-4b02-a4d6-9950edbc9ac9 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:48.326495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956b5f2-be12-4b02-a4d6-9950edbc9ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:48.326618 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956b5f2-be12-4b02-a4d6-9950edbc9ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:48.397636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-a956b5f2-be12-4b02-a4d6-9950edbc9ac9 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:48.398928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a956b5f2-be12-4b02-a4d6-9950edbc9ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:48.457750 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:48.501605 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 719/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:48.798492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a80db-a353-41cb-8ef1-719f059c6a0e None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:48.800695 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c16a80db-a353-41cb-8ef1-719f059c6a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:48.949571 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a80db-a353-41cb-8ef1-719f059c6a0e None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:49.030273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:49.070381 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d18c1-3b37-4413-9d4c-ecbefc248102 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:49.071024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d18c1-3b37-4413-9d4c-ecbefc248102 service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:49.071301 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d18c1-3b37-4413-9d4c-ecbefc248102 service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:49.071576 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d18c1-3b37-4413-9d4c-ecbefc248102 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:49.073186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4f9d18c1-3b37-4413-9d4c-ecbefc248102 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:49.164975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d18c1-3b37-4413-9d4c-ecbefc248102 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:49.166577 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d18c1-3b37-4413-9d4c-ecbefc248102 service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:49.185620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 697/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:50.337316 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e33345-9a6c-487b-b3e4-b183223e489a None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:50.337813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e33345-9a6c-487b-b3e4-b183223e489a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:50.337909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e33345-9a6c-487b-b3e4-b183223e489a None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:50.339290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 720/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:51:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:50.346674 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0f79d-bde9-47a1-992b-58d294547c4b None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:50.347235 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0f79d-bde9-47a1-992b-58d294547c4b None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:50.347235 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0f79d-bde9-47a1-992b-58d294547c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:50.425859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-08b0f79d-bde9-47a1-992b-58d294547c4b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:50.427104 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-08b0f79d-bde9-47a1-992b-58d294547c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:50.454159 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:50.525389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 698/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:50.675718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e33345-9a6c-487b-b3e4-b183223e489a None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:50.677235 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-71e33345-9a6c-487b-b3e4-b183223e489a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:50.826049 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e33345-9a6c-487b-b3e4-b183223e489a None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:50.867314 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:50.940616 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543e32f-f77a-46eb-b03e-926796fa9151 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:50.941222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543e32f-f77a-46eb-b03e-926796fa9151 service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:50.941499 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543e32f-f77a-46eb-b03e-926796fa9151 service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:50.941769 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543e32f-f77a-46eb-b03e-926796fa9151 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:50.943397 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-9543e32f-f77a-46eb-b03e-926796fa9151 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:51.035402 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543e32f-f77a-46eb-b03e-926796fa9151 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:51.037000 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543e32f-f77a-46eb-b03e-926796fa9151 service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:51.055088 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 721/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:52.200008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d5d33-baf5-485b-ad06-fd30cfd479e7 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:52.200610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d5d33-baf5-485b-ad06-fd30cfd479e7 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:52.200610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d5d33-baf5-485b-ad06-fd30cfd479e7 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:52.201882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 699/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:52.208381 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2b9b7-4c05-4227-b9ad-a48fe50a6e5f None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:52.208860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2b9b7-4c05-4227-b9ad-a48fe50a6e5f None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:52.208967 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2b9b7-4c05-4227-b9ad-a48fe50a6e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:52.263853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:52.286971 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-b8a2b9b7-4c05-4227-b9ad-a48fe50a6e5f None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:52.288173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b8a2b9b7-4c05-4227-b9ad-a48fe50a6e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:52.384662 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 722/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:52.536206 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d5d33-baf5-485b-ad06-fd30cfd479e7 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:52.537661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-5c1d5d33-baf5-485b-ad06-fd30cfd479e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:52.684495 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d5d33-baf5-485b-ad06-fd30cfd479e7 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:52.697487 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:52.803818 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15a49e-40c8-4c12-ad5c-dbf650bec56b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:52.804328 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15a49e-40c8-4c12-ad5c-dbf650bec56b service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:52.804605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15a49e-40c8-4c12-ad5c-dbf650bec56b service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:52.804881 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15a49e-40c8-4c12-ad5c-dbf650bec56b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:52.806477 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4c15a49e-40c8-4c12-ad5c-dbf650bec56b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:52.896595 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15a49e-40c8-4c12-ad5c-dbf650bec56b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:52.898196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15a49e-40c8-4c12-ad5c-dbf650bec56b service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:52.916742 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 700/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:54.032601 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970070e3-d8ca-4391-b5ab-d33f8161422f None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:54.033144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970070e3-d8ca-4391-b5ab-d33f8161422f None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:54.033144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970070e3-d8ca-4391-b5ab-d33f8161422f None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:54.034529 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 723/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:54.040944 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4b96c-1695-49fb-b92f-e56a3c680a94 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:54.041366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4b96c-1695-49fb-b92f-e56a3c680a94 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:54.041468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4b96c-1695-49fb-b92f-e56a3c680a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:54.060843 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:54.117842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-12e4b96c-1695-49fb-b92f-e56a3c680a94 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:54.119073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-12e4b96c-1695-49fb-b92f-e56a3c680a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:54.216309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 701/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:54.367740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970070e3-d8ca-4391-b5ab-d33f8161422f None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:54.369271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-970070e3-d8ca-4391-b5ab-d33f8161422f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:54.487279 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:54.524705 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970070e3-d8ca-4391-b5ab-d33f8161422f None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:54.638133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c4ea1-185f-4970-97ad-a4ef3a5a7e06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:54.638643 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c4ea1-185f-4970-97ad-a4ef3a5a7e06 service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:54.638943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c4ea1-185f-4970-97ad-a4ef3a5a7e06 service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:54.639211 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c4ea1-185f-4970-97ad-a4ef3a5a7e06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:54.640781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-922c4ea1-185f-4970-97ad-a4ef3a5a7e06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:54.715739 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:54.730160 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c4ea1-185f-4970-97ad-a4ef3a5a7e06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:54.731771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c4ea1-185f-4970-97ad-a4ef3a5a7e06 service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:54.752397 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 724/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:55.885865 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680079ea-c7df-4e25-9e1d-af5c9249f66b None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:55.886551 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680079ea-c7df-4e25-9e1d-af5c9249f66b None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:55.886551 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680079ea-c7df-4e25-9e1d-af5c9249f66b None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:55.887794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 702/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:55.894067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0ea64-a191-423f-aa12-4bad2f264674 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:55.894560 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0ea64-a191-423f-aa12-4bad2f264674 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:55.894654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0ea64-a191-423f-aa12-4bad2f264674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:55.970909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-b6a0ea64-a191-423f-aa12-4bad2f264674 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:55.972131 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b6a0ea64-a191-423f-aa12-4bad2f264674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:56.076063 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 725/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:51:56.228904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680079ea-c7df-4e25-9e1d-af5c9249f66b None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:56.230527 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-680079ea-c7df-4e25-9e1d-af5c9249f66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:56.315636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:56.378535 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680079ea-c7df-4e25-9e1d-af5c9249f66b None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:56.501390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ede62-6d66-4264-8313-9e4f791820c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:56.501920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ede62-6d66-4264-8313-9e4f791820c7 service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:56.502221 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ede62-6d66-4264-8313-9e4f791820c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:56.502494 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ede62-6d66-4264-8313-9e4f791820c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:56.504150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a89ede62-6d66-4264-8313-9e4f791820c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:56.556268 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:56.598716 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ede62-6d66-4264-8313-9e4f791820c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:56.600354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ede62-6d66-4264-8313-9e4f791820c7 service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:56.618300 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 703/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:57.693335 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5f1d9-add2-4a4f-a004-d0760b816669 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:57.693870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5f1d9-add2-4a4f-a004-d0760b816669 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:57.693870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5f1d9-add2-4a4f-a004-d0760b816669 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:57.695208 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 726/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:57.701515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cc0d5-9d0c-42ea-bc6b-d15d97b05765 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:57.701977 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cc0d5-9d0c-42ea-bc6b-d15d97b05765 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:57.702177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cc0d5-9d0c-42ea-bc6b-d15d97b05765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:57.778873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-5b2cc0d5-9d0c-42ea-bc6b-d15d97b05765 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:57.779951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-5b2cc0d5-9d0c-42ea-bc6b-d15d97b05765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:57.879912 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 704/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:51:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:58.045413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5f1d9-add2-4a4f-a004-d0760b816669 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:58.046884 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e3d5f1d9-add2-4a4f-a004-d0760b816669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:58.210855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:58.311175 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5f1d9-add2-4a4f-a004-d0760b816669 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:51:58.438370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c954eef-2de3-422e-9eee-952064a6f0b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:51:58.438994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c954eef-2de3-422e-9eee-952064a6f0b6 service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:58.439286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c954eef-2de3-422e-9eee-952064a6f0b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:58.439670 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c954eef-2de3-422e-9eee-952064a6f0b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:58.441306 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-6c954eef-2de3-422e-9eee-952064a6f0b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:58.448210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:58.529394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c954eef-2de3-422e-9eee-952064a6f0b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:51:58.531014 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c954eef-2de3-422e-9eee-952064a6f0b6 service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:51:58.549614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 727/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:51:59.613346 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82305232-d7af-4cd6-b7ab-5819891c495b None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:59.613876 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82305232-d7af-4cd6-b7ab-5819891c495b None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:59.613876 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82305232-d7af-4cd6-b7ab-5819891c495b None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:59.615253 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 705/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:51:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:51:59.621680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba569ae-0f5a-4ee5-8872-a9df238964f2 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:51:59.622130 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba569ae-0f5a-4ee5-8872-a9df238964f2 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:51:59.622257 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba569ae-0f5a-4ee5-8872-a9df238964f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:51:59.704591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-bba569ae-0f5a-4ee5-8872-a9df238964f2 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:51:59.705819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-bba569ae-0f5a-4ee5-8872-a9df238964f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:59.775014 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:51:59.804305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 728/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:51:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:51:59.958436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82305232-d7af-4cd6-b7ab-5819891c495b None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:51:59.959941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-82305232-d7af-4cd6-b7ab-5819891c495b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:51:59.978295 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:00.118589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82305232-d7af-4cd6-b7ab-5819891c495b None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:00.203434 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:00.229102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3c352-30f5-459b-bbef-8564b26bb4d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:00.229629 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3c352-30f5-459b-bbef-8564b26bb4d0 service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:00.229898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3c352-30f5-459b-bbef-8564b26bb4d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:00.230161 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3c352-30f5-459b-bbef-8564b26bb4d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:00.232016 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e2f3c352-30f5-459b-bbef-8564b26bb4d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:00.322416 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f3c352-30f5-459b-bbef-8564b26bb4d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:00.324074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f3c352-30f5-459b-bbef-8564b26bb4d0 service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:00.346945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 706/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:51:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:01.416033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371e94b-e4d2-4b1d-95db-91c79a81ca28 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:01.416637 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371e94b-e4d2-4b1d-95db-91c79a81ca28 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:01.416637 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371e94b-e4d2-4b1d-95db-91c79a81ca28 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:01.418010 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 729/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:01.424401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9c901-facd-4c09-9542-7071a95c1a6f None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:01.424854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9c901-facd-4c09-9542-7071a95c1a6f None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:01.424959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9c901-facd-4c09-9542-7071a95c1a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:01.499722 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-37d9c901-facd-4c09-9542-7071a95c1a6f None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:01.500934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-37d9c901-facd-4c09-9542-7071a95c1a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:01.545736 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:01.605086 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 707/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:52:01.756813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371e94b-e4d2-4b1d-95db-91c79a81ca28 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:01.758332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-4371e94b-e4d2-4b1d-95db-91c79a81ca28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:01.906378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371e94b-e4d2-4b1d-95db-91c79a81ca28 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:01.960705 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:02.021493 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e04a6-eb67-4013-b91e-8a5c1b7037d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:02.022098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e04a6-eb67-4013-b91e-8a5c1b7037d9 service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:02.022377 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e04a6-eb67-4013-b91e-8a5c1b7037d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:02.022647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e04a6-eb67-4013-b91e-8a5c1b7037d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:02.024268 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b75e04a6-eb67-4013-b91e-8a5c1b7037d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:02.114526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e04a6-eb67-4013-b91e-8a5c1b7037d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:02.116149 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e04a6-eb67-4013-b91e-8a5c1b7037d9 service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:02.133804 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 730/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:03.207169 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db5e12-2959-4e89-aa2b-c1999438f386 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:03.207753 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db5e12-2959-4e89-aa2b-c1999438f386 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:03.207753 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db5e12-2959-4e89-aa2b-c1999438f386 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:03.209094 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 708/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:03.215278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b0f5e-d061-4c0e-8564-82ed0fd24d24 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:03.215755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b0f5e-d061-4c0e-8564-82ed0fd24d24 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:03.215868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b0f5e-d061-4c0e-8564-82ed0fd24d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:03.293619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-636b0f5e-d061-4c0e-8564-82ed0fd24d24 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:03.294885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-636b0f5e-d061-4c0e-8564-82ed0fd24d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:03.296931 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:03.393064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 731/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:52:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:03.542138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db5e12-2959-4e89-aa2b-c1999438f386 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:03.543677 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b2db5e12-2959-4e89-aa2b-c1999438f386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:03.689211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db5e12-2959-4e89-aa2b-c1999438f386 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:03.716282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:03.802130 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a752b-2cb4-4a4a-b779-c4b75f1c1a37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:03.802651 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a752b-2cb4-4a4a-b779-c4b75f1c1a37 service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:03.802954 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a752b-2cb4-4a4a-b779-c4b75f1c1a37 service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:03.803225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a752b-2cb4-4a4a-b779-c4b75f1c1a37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:03.804824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2f5a752b-2cb4-4a4a-b779-c4b75f1c1a37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:03.896232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a752b-2cb4-4a4a-b779-c4b75f1c1a37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:03.897830 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5a752b-2cb4-4a4a-b779-c4b75f1c1a37 service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:03.916349 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 709/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:06.007299 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d164ab4-96d3-4f84-abd9-3bed30a13b4a None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:06.007848 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d164ab4-96d3-4f84-abd9-3bed30a13b4a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:06.007848 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d164ab4-96d3-4f84-abd9-3bed30a13b4a None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:06.009114 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 732/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:06.014643 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca68c65-cc7d-4108-b93d-28b548a683f0 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:06.015117 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca68c65-cc7d-4108-b93d-28b548a683f0 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:06.015212 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca68c65-cc7d-4108-b93d-28b548a683f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:06.056886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:06.084524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-2ca68c65-cc7d-4108-b93d-28b548a683f0 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:06.085731 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2ca68c65-cc7d-4108-b93d-28b548a683f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:06.180963 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 710/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:06.462426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d164ab4-96d3-4f84-abd9-3bed30a13b4a None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:06.464030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-1d164ab4-96d3-4f84-abd9-3bed30a13b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:06.567327 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d164ab4-96d3-4f84-abd9-3bed30a13b4a None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:06.600482 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:06.641475 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd6154-7be9-4aa5-9d04-f55bd5b6b5fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:06.642083 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd6154-7be9-4aa5-9d04-f55bd5b6b5fc service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:06.642879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd6154-7be9-4aa5-9d04-f55bd5b6b5fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:06.643155 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd6154-7be9-4aa5-9d04-f55bd5b6b5fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:06.644763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-cafd6154-7be9-4aa5-9d04-f55bd5b6b5fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:06.753045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd6154-7be9-4aa5-9d04-f55bd5b6b5fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:06.754469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd6154-7be9-4aa5-9d04-f55bd5b6b5fc service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:06.772222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 733/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:09.109494 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1eab5-4480-45fe-a6ae-9d52308155fe None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:09.110181 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1eab5-4480-45fe-a6ae-9d52308155fe None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:09.110181 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1eab5-4480-45fe-a6ae-9d52308155fe None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:09.111456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 711/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:09.118009 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e5186-01c0-4281-adea-9e297344f216 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:09.118509 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e5186-01c0-4281-adea-9e297344f216 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:09.118605 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e5186-01c0-4281-adea-9e297344f216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:09.185336 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-a15e5186-01c0-4281-adea-9e297344f216 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:09.186557 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a15e5186-01c0-4281-adea-9e297344f216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:09.199318 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:09.284283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 734/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:09.571693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1eab5-4480-45fe-a6ae-9d52308155fe None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:09.573246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3eb1eab5-4480-45fe-a6ae-9d52308155fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:09.677108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1eab5-4480-45fe-a6ae-9d52308155fe None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:09.691259 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:09.757237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3e8b6-d221-4a3f-a8a6-b57e94ed41ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:09.757846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3e8b6-d221-4a3f-a8a6-b57e94ed41ca service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:09.758125 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3e8b6-d221-4a3f-a8a6-b57e94ed41ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:09.758399 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3e8b6-d221-4a3f-a8a6-b57e94ed41ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:09.760033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-cde3e8b6-d221-4a3f-a8a6-b57e94ed41ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:09.873668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3e8b6-d221-4a3f-a8a6-b57e94ed41ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:09.875164 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3e8b6-d221-4a3f-a8a6-b57e94ed41ca service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:09.895187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 712/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:12.470637 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d3345-d9cb-4ed3-bcec-3e4b2761d1dd None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:12.471742 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d3345-d9cb-4ed3-bcec-3e4b2761d1dd None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:12.471858 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d3345-d9cb-4ed3-bcec-3e4b2761d1dd None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:12.473136 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 735/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:12.478496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718349d0-79a1-407e-a91e-f451626ecf1e None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:12.478948 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718349d0-79a1-407e-a91e-f451626ecf1e None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:12.479066 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718349d0-79a1-407e-a91e-f451626ecf1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:12.525640 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:12.544657 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-718349d0-79a1-407e-a91e-f451626ecf1e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:12.545879 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-718349d0-79a1-407e-a91e-f451626ecf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:12.645035 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 713/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:52:12.928981 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d3345-d9cb-4ed3-bcec-3e4b2761d1dd None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:12.930513 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ea0d3345-d9cb-4ed3-bcec-3e4b2761d1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:13.034132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d3345-d9cb-4ed3-bcec-3e4b2761d1dd None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:13.074254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:13.107120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c9f15-7145-4978-bfb9-843ee011a79c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:13.107747 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c9f15-7145-4978-bfb9-843ee011a79c service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:13.108018 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c9f15-7145-4978-bfb9-843ee011a79c service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:13.108282 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c9f15-7145-4978-bfb9-843ee011a79c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:13.109847 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e43c9f15-7145-4978-bfb9-843ee011a79c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:13.216997 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c9f15-7145-4978-bfb9-843ee011a79c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:13.218343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c9f15-7145-4978-bfb9-843ee011a79c service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:13.236160 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 736/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:15.555069 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b928e69-100e-4609-ba52-92214c26b5b7 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:15.555896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b928e69-100e-4609-ba52-92214c26b5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:15.555896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b928e69-100e-4609-ba52-92214c26b5b7 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:15.557117 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 714/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:15.565838 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b4a20-9947-4069-a7f5-7b0611c77ed4 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:15.566372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b4a20-9947-4069-a7f5-7b0611c77ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:15.566677 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b4a20-9947-4069-a7f5-7b0611c77ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:15.654415 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-1d8b4a20-9947-4069-a7f5-7b0611c77ed4 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:15.655588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-1d8b4a20-9947-4069-a7f5-7b0611c77ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:15.677077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:15.758961 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 737/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:16.069135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b928e69-100e-4609-ba52-92214c26b5b7 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:16.070678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4b928e69-100e-4609-ba52-92214c26b5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:16.172910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b928e69-100e-4609-ba52-92214c26b5b7 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:16.194829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:16.249003 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfcbcc-bf6f-4310-a4f5-d89db148296f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:16.249611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfcbcc-bf6f-4310-a4f5-d89db148296f service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:16.249890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfcbcc-bf6f-4310-a4f5-d89db148296f service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:16.250162 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfcbcc-bf6f-4310-a4f5-d89db148296f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:16.251791 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-abcfcbcc-bf6f-4310-a4f5-d89db148296f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:16.365134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfcbcc-bf6f-4310-a4f5-d89db148296f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:16.366603 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfcbcc-bf6f-4310-a4f5-d89db148296f service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:16.385544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 715/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:18.618416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a1758-0f01-4b07-917b-8fc155cae439 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:18.619016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a1758-0f01-4b07-917b-8fc155cae439 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:18.619016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a1758-0f01-4b07-917b-8fc155cae439 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:18.620368 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 738/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:18.625617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f37ad-837d-4fac-b1cc-8e5d7d781b07 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:18.626060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f37ad-837d-4fac-b1cc-8e5d7d781b07 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:18.626189 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f37ad-837d-4fac-b1cc-8e5d7d781b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:18.690493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:18.693913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-3d3f37ad-837d-4fac-b1cc-8e5d7d781b07 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:18.695347 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3d3f37ad-837d-4fac-b1cc-8e5d7d781b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:18.790809 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 716/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:19.072628 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a1758-0f01-4b07-917b-8fc155cae439 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:19.074145 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-092a1758-0f01-4b07-917b-8fc155cae439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:19.176738 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a1758-0f01-4b07-917b-8fc155cae439 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:19.226449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:19.250905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ee423-df6e-4533-b495-a1d5f477981a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:19.251517 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ee423-df6e-4533-b495-a1d5f477981a service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:19.251847 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ee423-df6e-4533-b495-a1d5f477981a service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:19.252113 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ee423-df6e-4533-b495-a1d5f477981a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:19.253704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-869ee423-df6e-4533-b495-a1d5f477981a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:19.365466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ee423-df6e-4533-b495-a1d5f477981a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:19.366927 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ee423-df6e-4533-b495-a1d5f477981a service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:19.386856 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 739/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:21.619411 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d9539-b089-4e89-8146-b5d8979b54b2 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:21.619950 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d9539-b089-4e89-8146-b5d8979b54b2 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:21.620070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d9539-b089-4e89-8146-b5d8979b54b2 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:21.621484 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 717/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:21.627763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad824f9-d066-4a1b-9b63-2e4c1b44e0d8 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:21.628239 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad824f9-d066-4a1b-9b63-2e4c1b44e0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:21.628336 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad824f9-d066-4a1b-9b63-2e4c1b44e0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:21.693765 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-4ad824f9-d066-4a1b-9b63-2e4c1b44e0d8 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:21.694995 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-4ad824f9-d066-4a1b-9b63-2e4c1b44e0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:21.723905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:21.796333 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 740/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:22.082383 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d9539-b089-4e89-8146-b5d8979b54b2 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:22.084000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e75d9539-b089-4e89-8146-b5d8979b54b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:22.193057 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d9539-b089-4e89-8146-b5d8979b54b2 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:22.223207 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:22.269360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd545a3-8eed-40e6-b8a7-ebecc8b6321c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:22.269964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd545a3-8eed-40e6-b8a7-ebecc8b6321c service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:22.270242 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd545a3-8eed-40e6-b8a7-ebecc8b6321c service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:22.270507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd545a3-8eed-40e6-b8a7-ebecc8b6321c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:22.272120 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-bbd545a3-8eed-40e6-b8a7-ebecc8b6321c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:22.384147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd545a3-8eed-40e6-b8a7-ebecc8b6321c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:22.385587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd545a3-8eed-40e6-b8a7-ebecc8b6321c service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:22.404425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 718/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:24.653669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0630de-d48c-4dc3-b3cb-427e6498b49b None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:24.654209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0630de-d48c-4dc3-b3cb-427e6498b49b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:24.654209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0630de-d48c-4dc3-b3cb-427e6498b49b None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:24.655593 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 741/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:24.661110 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f61fa-22f4-4d23-95c9-41d99b6a318c None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:24.661521 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f61fa-22f4-4d23-95c9-41d99b6a318c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:24.661649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f61fa-22f4-4d23-95c9-41d99b6a318c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:24.727173 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-626f61fa-22f4-4d23-95c9-41d99b6a318c None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:24.728369 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-626f61fa-22f4-4d23-95c9-41d99b6a318c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:24.737131 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:24.822913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 719/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:52:25.116880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0630de-d48c-4dc3-b3cb-427e6498b49b None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:25.124443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-9c0630de-d48c-4dc3-b3cb-427e6498b49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:25.234971 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0630de-d48c-4dc3-b3cb-427e6498b49b None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:25.290114 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:25.308261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74541d00-8acb-4713-b56c-29f9601cd7d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:25.308884 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74541d00-8acb-4713-b56c-29f9601cd7d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:25.309156 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74541d00-8acb-4713-b56c-29f9601cd7d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:25.309422 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74541d00-8acb-4713-b56c-29f9601cd7d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:25.311028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-74541d00-8acb-4713-b56c-29f9601cd7d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:25.420740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74541d00-8acb-4713-b56c-29f9601cd7d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:25.422161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74541d00-8acb-4713-b56c-29f9601cd7d2 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:25.440995 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 742/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:27.763787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deab2de-7ca1-49ed-9b3a-c46a42b8dbbd None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:27.764391 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deab2de-7ca1-49ed-9b3a-c46a42b8dbbd None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:27.764391 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deab2de-7ca1-49ed-9b3a-c46a42b8dbbd None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:27.765611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 720/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:27.770939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b872a3-e647-4447-ab54-8b24056ac06c None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:27.771391 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b872a3-e647-4447-ab54-8b24056ac06c None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:27.771498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b872a3-e647-4447-ab54-8b24056ac06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:27.836554 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-80b872a3-e647-4447-ab54-8b24056ac06c None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:27.837745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-80b872a3-e647-4447-ab54-8b24056ac06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:27.879088 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:27.939679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 743/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:28.221954 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deab2de-7ca1-49ed-9b3a-c46a42b8dbbd None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:28.223524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2deab2de-7ca1-49ed-9b3a-c46a42b8dbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:28.328503 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deab2de-7ca1-49ed-9b3a-c46a42b8dbbd None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:28.367579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:28.404743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0263d01-894f-4cda-93ce-0f258138ef76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:28.405356 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0263d01-894f-4cda-93ce-0f258138ef76 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:28.405634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0263d01-894f-4cda-93ce-0f258138ef76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:28.405904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0263d01-894f-4cda-93ce-0f258138ef76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:28.407536 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e0263d01-894f-4cda-93ce-0f258138ef76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:28.522772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0263d01-894f-4cda-93ce-0f258138ef76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:28.524106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0263d01-894f-4cda-93ce-0f258138ef76 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:28.542651 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 721/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:52:30.745010 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64cabbd-ff10-4b9c-9215-f22910552969 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:30.745574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64cabbd-ff10-4b9c-9215-f22910552969 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:30.745574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64cabbd-ff10-4b9c-9215-f22910552969 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:30.746929 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 744/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:30.752244 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44513ee0-f388-413b-b772-92c552631f89 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:30.752650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44513ee0-f388-413b-b772-92c552631f89 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:30.752810 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44513ee0-f388-413b-b772-92c552631f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:30.818033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-44513ee0-f388-413b-b772-92c552631f89 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:30.819281 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-44513ee0-f388-413b-b772-92c552631f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:30.837703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:30.916664 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 722/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:31.204435 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64cabbd-ff10-4b9c-9215-f22910552969 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:31.206095 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-f64cabbd-ff10-4b9c-9215-f22910552969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:31.310838 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64cabbd-ff10-4b9c-9215-f22910552969 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:31.381705 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:31.389027 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ffbf92-5b56-4eb2-ae6e-ba5b0c424601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:31.389721 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffbf92-5b56-4eb2-ae6e-ba5b0c424601 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:31.390002 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffbf92-5b56-4eb2-ae6e-ba5b0c424601 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:31.390274 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ffbf92-5b56-4eb2-ae6e-ba5b0c424601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:31.391981 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-82ffbf92-5b56-4eb2-ae6e-ba5b0c424601 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:31.503456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ffbf92-5b56-4eb2-ae6e-ba5b0c424601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:31.504808 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ffbf92-5b56-4eb2-ae6e-ba5b0c424601 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:31.525074 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 745/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:33.694987 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760c926-9d98-4523-a3a5-a1cf7dd3adb6 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:33.695499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760c926-9d98-4523-a3a5-a1cf7dd3adb6 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:33.695499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760c926-9d98-4523-a3a5-a1cf7dd3adb6 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:33.696785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 723/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:33.702030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05485f06-006c-40b6-aca7-f161851c525d None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:33.702469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05485f06-006c-40b6-aca7-f161851c525d None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:33.702610 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05485f06-006c-40b6-aca7-f161851c525d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:33.768244 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-05485f06-006c-40b6-aca7-f161851c525d None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:33.769463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-05485f06-006c-40b6-aca7-f161851c525d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:33.817922 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:33.867398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 746/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:34.155475 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760c926-9d98-4523-a3a5-a1cf7dd3adb6 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:34.157044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a760c926-9d98-4523-a3a5-a1cf7dd3adb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:34.259757 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760c926-9d98-4523-a3a5-a1cf7dd3adb6 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:34.309100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:34.336964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a6d89-1455-4be4-97e1-6ade85f85e82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:34.337581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a6d89-1455-4be4-97e1-6ade85f85e82 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:34.337860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a6d89-1455-4be4-97e1-6ade85f85e82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:34.338128 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a6d89-1455-4be4-97e1-6ade85f85e82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:34.339764 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4f4a6d89-1455-4be4-97e1-6ade85f85e82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:34.450381 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a6d89-1455-4be4-97e1-6ade85f85e82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:34.451747 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a6d89-1455-4be4-97e1-6ade85f85e82 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:34.470867 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 724/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:43.865826 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=22 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:44.467796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=66 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:54.647410 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f6281-a660-4b36-bfa9-57d224d096e6 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:54.647988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f6281-a660-4b36-bfa9-57d224d096e6 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:54.647988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f6281-a660-4b36-bfa9-57d224d096e6 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:54.649296 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 747/1471] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:54.654831 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51011f-f174-4c08-99d5-487f79347bb8 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:54.655306 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51011f-f174-4c08-99d5-487f79347bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:54.655405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51011f-f174-4c08-99d5-487f79347bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:54.720996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-bf51011f-f174-4c08-99d5-487f79347bb8 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:54.722215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-bf51011f-f174-4c08-99d5-487f79347bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:54.818753 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 725/1472] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:55.126018 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efcb86-2da4-4fc4-af69-ed6b6e82df12 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:55.126301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efcb86-2da4-4fc4-af69-ed6b6e82df12 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:55.126577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09efcb86-2da4-4fc4-af69-ed6b6e82df12 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:55.127956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 748/1473] 10.222.0.26 () {54 vars in 962 bytes} [Thu Sep 22 14:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:55.133148 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68606397-2fd6-42c3-b432-6ea7f34b2cfb None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:55.133433 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68606397-2fd6-42c3-b432-6ea7f34b2cfb None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:55.133706 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68606397-2fd6-42c3-b432-6ea7f34b2cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:55.209585 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-68606397-2fd6-42c3-b432-6ea7f34b2cfb None None] MFA Rules not processed for user `bb5248cb1c39412680cce0d9891fe1ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:55.210680 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-68606397-2fd6-42c3-b432-6ea7f34b2cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:55.232509 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:55.289140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 726/1474] 10.222.0.26 () {58 vars in 1059 bytes} [Thu Sep 22 14:52:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:55.296939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efcb86-2da4-4fc4-af69-ed6b6e82df12 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:55.298551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-09efcb86-2da4-4fc4-af69-ed6b6e82df12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:55.407836 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09efcb86-2da4-4fc4-af69-ed6b6e82df12 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:55.482828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ac74e-050c-4121-a559-b6a2566adf41 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5248cb1c39412680cce0d9891fe1ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:55.483363 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ac74e-050c-4121-a559-b6a2566adf41 service glance] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:55.483641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ac74e-050c-4121-a559-b6a2566adf41 service glance] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:55.483913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ac74e-050c-4121-a559-b6a2566adf41 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:55.485528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-962ac74e-050c-4121-a559-b6a2566adf41 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:55.539869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:55.596218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ac74e-050c-4121-a559-b6a2566adf41 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:55.597650 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ac74e-050c-4121-a559-b6a2566adf41 service glance] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:55.616042 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 749/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:52:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:52:57.898136 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66753c43-e4c6-49c7-8af3-8f21e6fcdac0 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:57.898769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66753c43-e4c6-49c7-8af3-8f21e6fcdac0 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:57.898769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66753c43-e4c6-49c7-8af3-8f21e6fcdac0 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:57.900071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 727/1476] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:52:57.905473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa35870-3552-4b8b-9723-d844584d85a1 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:57.905935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa35870-3552-4b8b-9723-d844584d85a1 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:57.906043 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa35870-3552-4b8b-9723-d844584d85a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:57.971245 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-8fa35870-3552-4b8b-9723-d844584d85a1 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:52:57.972466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-8fa35870-3552-4b8b-9723-d844584d85a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:58.070964 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 750/1477] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:52:58.378771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66753c43-e4c6-49c7-8af3-8f21e6fcdac0 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:52:58.380296 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-66753c43-e4c6-49c7-8af3-8f21e6fcdac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:52:58.508715 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66753c43-e4c6-49c7-8af3-8f21e6fcdac0 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:52:58.558139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:52:58.602352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b044f-84a9-4f36-a187-e3cf8b21aa01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:52:58.602996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b044f-84a9-4f36-a187-e3cf8b21aa01 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:52:58.603272 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b044f-84a9-4f36-a187-e3cf8b21aa01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:52:58.603540 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b044f-84a9-4f36-a187-e3cf8b21aa01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:52:58.604177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b044f-84a9-4f36-a187-e3cf8b21aa01 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:52:58.605965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: WARNING py.warnings [None req-d27b044f-84a9-4f36-a187-e3cf8b21aa01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:52:58.605965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: warnings.warn(msg) Sep 22 14:52:58.605965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: Sep 22 14:52:58.607145 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b044f-84a9-4f36-a187-e3cf8b21aa01 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:52:58.626702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 728/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Sep 22 14:52:58 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:53:00.854406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be48fb8-bcc5-4ed1-998d-67fc48f5f9bf None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:00.855020 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be48fb8-bcc5-4ed1-998d-67fc48f5f9bf None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:00.855127 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be48fb8-bcc5-4ed1-998d-67fc48f5f9bf None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:00.856402 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 751/1479] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:53:00.861730 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bafce-f54a-48c5-92cb-36f6725cf807 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:00.862130 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bafce-f54a-48c5-92cb-36f6725cf807 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:00.862303 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bafce-f54a-48c5-92cb-36f6725cf807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:00.929499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-419bafce-f54a-48c5-92cb-36f6725cf807 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:53:00.930748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-419bafce-f54a-48c5-92cb-36f6725cf807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:01.033629 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 729/1480] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:53:01.328101 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be48fb8-bcc5-4ed1-998d-67fc48f5f9bf None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:53:01.329302 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3be48fb8-bcc5-4ed1-998d-67fc48f5f9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:01.339201 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:01.476154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be48fb8-bcc5-4ed1-998d-67fc48f5f9bf None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:53:01.556190 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:01.590383 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e0160-c3b7-4114-ae3d-4a13921fdc20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:53:01.591035 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e0160-c3b7-4114-ae3d-4a13921fdc20 service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:01.591313 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e0160-c3b7-4114-ae3d-4a13921fdc20 service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:01.591629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e0160-c3b7-4114-ae3d-4a13921fdc20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:01.593195 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-863e0160-c3b7-4114-ae3d-4a13921fdc20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:01.686095 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e0160-c3b7-4114-ae3d-4a13921fdc20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:53:01.687715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e0160-c3b7-4114-ae3d-4a13921fdc20 service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:53:01.705242 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 752/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:53:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:53:04.105087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e62ee-4eba-46e6-99e3-1981e9bf9bff None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:04.105660 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e62ee-4eba-46e6-99e3-1981e9bf9bff None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:04.105660 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e62ee-4eba-46e6-99e3-1981e9bf9bff None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:04.107018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 730/1482] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:53:04.113022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08d45a-259c-4490-a4e2-0ee6cce129f3 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:04.113476 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08d45a-259c-4490-a4e2-0ee6cce129f3 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:04.113581 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08d45a-259c-4490-a4e2-0ee6cce129f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:04.178945 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-8f08d45a-259c-4490-a4e2-0ee6cce129f3 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:53:04.180173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-8f08d45a-259c-4490-a4e2-0ee6cce129f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:04.233641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:04.287360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 753/1483] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:53:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:53:04.576552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e62ee-4eba-46e6-99e3-1981e9bf9bff None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:53:04.578123 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c09e62ee-4eba-46e6-99e3-1981e9bf9bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:04.601974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:04.681331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e62ee-4eba-46e6-99e3-1981e9bf9bff None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:53:04.753947 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6695f27-8d09-4a07-b9dd-79b263f4c5a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:53:04.754554 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6695f27-8d09-4a07-b9dd-79b263f4c5a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:04.754868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6695f27-8d09-4a07-b9dd-79b263f4c5a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:04.755141 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6695f27-8d09-4a07-b9dd-79b263f4c5a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:04.756822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c6695f27-8d09-4a07-b9dd-79b263f4c5a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:04.949626 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:04.986103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6695f27-8d09-4a07-b9dd-79b263f4c5a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:53:04.987599 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6695f27-8d09-4a07-b9dd-79b263f4c5a9 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:53:05.006416 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 731/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:53:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:53:07.224540 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27f0a0-ed4f-4412-adae-001d22242816 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:07.225113 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27f0a0-ed4f-4412-adae-001d22242816 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:07.225113 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc27f0a0-ed4f-4412-adae-001d22242816 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:07.226428 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 754/1485] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:53:07.231699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3d34-4804-4093-b40d-2d59797b7d93 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:07.232108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3d34-4804-4093-b40d-2d59797b7d93 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:07.232260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3d34-4804-4093-b40d-2d59797b7d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:07.297855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-3b9f3d34-4804-4093-b40d-2d59797b7d93 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:53:07.299105 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3b9f3d34-4804-4093-b40d-2d59797b7d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:07.395072 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 732/1486] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:53:07.683279 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27f0a0-ed4f-4412-adae-001d22242816 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:53:07.684816 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-cc27f0a0-ed4f-4412-adae-001d22242816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:07.788295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc27f0a0-ed4f-4412-adae-001d22242816 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:53:07.861461 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:07.863018 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036d64c-a51c-4da2-ae97-5cc1aee899e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:53:07.863922 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036d64c-a51c-4da2-ae97-5cc1aee899e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:07.864283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036d64c-a51c-4da2-ae97-5cc1aee899e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:07.864631 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036d64c-a51c-4da2-ae97-5cc1aee899e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:07.867204 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a036d64c-a51c-4da2-ae97-5cc1aee899e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:07.977084 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036d64c-a51c-4da2-ae97-5cc1aee899e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:53:07.978438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036d64c-a51c-4da2-ae97-5cc1aee899e3 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:53:07.996667 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 755/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:53:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:53:10.253321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeddd25-badc-4ab3-a6a7-3e75e3ba21de None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:10.253894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeddd25-badc-4ab3-a6a7-3e75e3ba21de None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:10.253894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeddd25-badc-4ab3-a6a7-3e75e3ba21de None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:10.255258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 733/1488] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:53:10.260742 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9d88f-ea52-48ad-8fd5-9648dfbf8630 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:10.261194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9d88f-ea52-48ad-8fd5-9648dfbf8630 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:10.261299 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9d88f-ea52-48ad-8fd5-9648dfbf8630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:10.331331 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-8bb9d88f-ea52-48ad-8fd5-9648dfbf8630 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:53:10.332568 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-8bb9d88f-ea52-48ad-8fd5-9648dfbf8630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:10.386804 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:10.430559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 756/1489] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:53:10.721702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeddd25-badc-4ab3-a6a7-3e75e3ba21de None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:53:10.723269 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4eeddd25-badc-4ab3-a6a7-3e75e3ba21de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:10.759206 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:10.827648 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeddd25-badc-4ab3-a6a7-3e75e3ba21de None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:53:10.901651 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338b361-a019-4089-a015-6718b3688af4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:53:10.902261 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338b361-a019-4089-a015-6718b3688af4 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:10.902541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338b361-a019-4089-a015-6718b3688af4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:10.902875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338b361-a019-4089-a015-6718b3688af4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:10.904470 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b338b361-a019-4089-a015-6718b3688af4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:10.985188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:11.015399 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338b361-a019-4089-a015-6718b3688af4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:53:11.016829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338b361-a019-4089-a015-6718b3688af4 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:53:11.037964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 734/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:53:13.445190 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2adbc6-f627-4a46-9d40-5de09c21d4fc None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:13.445729 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2adbc6-f627-4a46-9d40-5de09c21d4fc None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:13.445729 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2adbc6-f627-4a46-9d40-5de09c21d4fc None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:13.447108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 757/1491] 10.222.0.26 () {54 vars in 927 bytes} [Thu Sep 22 14:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:53:13.453190 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342bca1-d00a-4905-80de-c4ebbbbb8494 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:13.453621 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342bca1-d00a-4905-80de-c4ebbbbb8494 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:13.453727 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342bca1-d00a-4905-80de-c4ebbbbb8494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:13.518455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-6342bca1-d00a-4905-80de-c4ebbbbb8494 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:53:13.519675 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-6342bca1-d00a-4905-80de-c4ebbbbb8494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:13.615787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 735/1492] 10.222.0.26 () {58 vars in 1024 bytes} [Thu Sep 22 14:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:53:13.914644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2adbc6-f627-4a46-9d40-5de09c21d4fc None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:53:13.916232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-2e2adbc6-f627-4a46-9d40-5de09c21d4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:14.029152 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2adbc6-f627-4a46-9d40-5de09c21d4fc None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:53:14.103526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b3551-862b-4f68-89d4-d415ad17848a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:53:14.104128 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b3551-862b-4f68-89d4-d415ad17848a service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:53:14.104411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b3551-862b-4f68-89d4-d415ad17848a service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:53:14.104682 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b3551-862b-4f68-89d4-d415ad17848a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:53:14.106279 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ed9b3551-862b-4f68-89d4-d415ad17848a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:53:14.115390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:14.217405 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b3551-862b-4f68-89d4-d415ad17848a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:53:14.219518 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b3551-862b-4f68-89d4-d415ad17848a service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:53:14.237494 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 758/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:53:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:53:23.613093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=82 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:53:24.235251 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=50 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:34.929102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b77c4-2bb0-48ed-9b20-9a708f06341d None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:34.929724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b77c4-2bb0-48ed-9b20-9a708f06341d None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:34.929724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b77c4-2bb0-48ed-9b20-9a708f06341d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:35.005883 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-d33b77c4-2bb0-48ed-9b20-9a708f06341d None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:54:35.007390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d33b77c4-2bb0-48ed-9b20-9a708f06341d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:35.106958 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 736/1494] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:54:35.115072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71171724-18dd-40fd-9e38-3ac123af032a None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:35.115588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71171724-18dd-40fd-9e38-3ac123af032a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:35.115735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71171724-18dd-40fd-9e38-3ac123af032a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:35.191501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-71171724-18dd-40fd-9e38-3ac123af032a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:54:35.192617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-71171724-18dd-40fd-9e38-3ac123af032a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:35.286501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 759/1495] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:54:35.291295 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b77c4-2bb0-48ed-9b20-9a708f06341d None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:35.292481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d33b77c4-2bb0-48ed-9b20-9a708f06341d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:35.304132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b77c4-2bb0-48ed-9b20-9a708f06341d None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:35.367022 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:35.400412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbb3a5-5df5-443d-b84b-eff1ab855c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:35.401013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbb3a5-5df5-443d-b84b-eff1ab855c01 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:35.401290 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbb3a5-5df5-443d-b84b-eff1ab855c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:35.401558 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbb3a5-5df5-443d-b84b-eff1ab855c01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:35.402187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbb3a5-5df5-443d-b84b-eff1ab855c01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:54:35.402513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbb3a5-5df5-443d-b84b-eff1ab855c01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:35.403613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: WARNING py.warnings [None req-89fbb3a5-5df5-443d-b84b-eff1ab855c01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:54:35.403613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: warnings.warn(msg) Sep 22 14:54:35.403613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: Sep 22 14:54:35.404483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbb3a5-5df5-443d-b84b-eff1ab855c01 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:35.415702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 737/1496] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:54:35.424631 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71171724-18dd-40fd-9e38-3ac123af032a None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:35.434210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71171724-18dd-40fd-9e38-3ac123af032a None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:35.497667 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:35.531303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbeaef-f1a0-42a9-becd-d16995eb1905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:35.531900 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbeaef-f1a0-42a9-becd-d16995eb1905 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:35.532171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbeaef-f1a0-42a9-becd-d16995eb1905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:35.532436 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadbeaef-f1a0-42a9-becd-d16995eb1905 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:35.533065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbeaef-f1a0-42a9-becd-d16995eb1905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:35.534825 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: WARNING py.warnings [None req-dadbeaef-f1a0-42a9-becd-d16995eb1905 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:54:35.534825 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: warnings.warn(msg) Sep 22 14:54:35.534825 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: Sep 22 14:54:35.536009 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadbeaef-f1a0-42a9-becd-d16995eb1905 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:35.591298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 760/1497] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:54:35 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:54:35.595761 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbb3a5-5df5-443d-b84b-eff1ab855c01 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:35.597319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-89fbb3a5-5df5-443d-b84b-eff1ab855c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:35.630852 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbb3a5-5df5-443d-b84b-eff1ab855c01 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:35.724997 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0012f0-65d9-4bc5-a029-707cce0fb704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:35.725628 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0012f0-65d9-4bc5-a029-707cce0fb704 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:35.725900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0012f0-65d9-4bc5-a029-707cce0fb704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:35.726161 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0012f0-65d9-4bc5-a029-707cce0fb704 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:35.726826 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0012f0-65d9-4bc5-a029-707cce0fb704 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:35.727827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: WARNING py.warnings [None req-fb0012f0-65d9-4bc5-a029-707cce0fb704 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:54:35.727827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: warnings.warn(msg) Sep 22 14:54:35.727827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: Sep 22 14:54:35.729008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0012f0-65d9-4bc5-a029-707cce0fb704 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:35.771001 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 738/1498] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:54:35 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:54:35.775221 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbeaef-f1a0-42a9-becd-d16995eb1905 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:35.776748 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-dadbeaef-f1a0-42a9-becd-d16995eb1905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:35.838654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadbeaef-f1a0-42a9-becd-d16995eb1905 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:35.926481 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:35.939530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b952fd7-5939-4ff4-a98d-80e93437af37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:35.940445 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b952fd7-5939-4ff4-a98d-80e93437af37 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:35.940855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b952fd7-5939-4ff4-a98d-80e93437af37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:35.941255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b952fd7-5939-4ff4-a98d-80e93437af37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:35.942238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b952fd7-5939-4ff4-a98d-80e93437af37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:35.943896 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: WARNING py.warnings [None req-7b952fd7-5939-4ff4-a98d-80e93437af37 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:54:35.943896 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: warnings.warn(msg) Sep 22 14:54:35.943896 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: Sep 22 14:54:35.945291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b952fd7-5939-4ff4-a98d-80e93437af37 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:35.965564 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 761/1499] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:54:35.970217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0012f0-65d9-4bc5-a029-707cce0fb704 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:35.971828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-fb0012f0-65d9-4bc5-a029-707cce0fb704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:35.983103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0012f0-65d9-4bc5-a029-707cce0fb704 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:36.009273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:36.080150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4a1b3-b725-4115-b5ad-2c3009915293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:36.080755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4a1b3-b725-4115-b5ad-2c3009915293 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:36.081036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4a1b3-b725-4115-b5ad-2c3009915293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:36.081303 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4a1b3-b725-4115-b5ad-2c3009915293 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:36.081937 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4a1b3-b725-4115-b5ad-2c3009915293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:36.083000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: WARNING py.warnings [None req-84c4a1b3-b725-4115-b5ad-2c3009915293 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:54:36.083000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: warnings.warn(msg) Sep 22 14:54:36.083000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: Sep 22 14:54:36.083878 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4a1b3-b725-4115-b5ad-2c3009915293 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:36.097098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 739/1500] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:54:36.101369 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b952fd7-5939-4ff4-a98d-80e93437af37 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:36.102932 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7b952fd7-5939-4ff4-a98d-80e93437af37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:36.136015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b952fd7-5939-4ff4-a98d-80e93437af37 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:36.231406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9b891-6462-4530-b4a2-306842eea224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:36.232070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9b891-6462-4530-b4a2-306842eea224 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:36.232342 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9b891-6462-4530-b4a2-306842eea224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:36.232608 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9b891-6462-4530-b4a2-306842eea224 admin admin] PATH_INFO: `/v3/projects/71d456a9f86540dbba2fc5bc83d3e139/users/fe48b7f3711541ee94333b5458dab84f/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:36.255241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9b891-6462-4530-b4a2-306842eea224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71d456a9f86540dbba2fc5bc83d3e139, user_id=fe48b7f3711541ee94333b5458dab84f, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:36.256263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: WARNING py.warnings [None req-74f9b891-6462-4530-b4a2-306842eea224 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:54:36.256263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: warnings.warn(msg) Sep 22 14:54:36.256263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: Sep 22 14:54:36.258199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9b891-6462-4530-b4a2-306842eea224 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:36.283212 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 762/1501] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:54:36 2022] PUT /identity/v3/projects/71d456a9f86540dbba2fc5bc83d3e139/users/fe48b7f3711541ee94333b5458dab84f/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:54:36.296196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c168deb-7411-4276-9ea2-e8a10fac2148 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:36.296477 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c168deb-7411-4276-9ea2-e8a10fac2148 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:36.296750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c168deb-7411-4276-9ea2-e8a10fac2148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:36.357099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-9c168deb-7411-4276-9ea2-e8a10fac2148 None None] MFA Rules not processed for user `fe48b7f3711541ee94333b5458dab84f`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:54:36.358246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-9c168deb-7411-4276-9ea2-e8a10fac2148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:36.436769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 740/1502] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:54:36.451392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fc395-959f-4965-afd3-f4de043fa01a None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:36.451392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fc395-959f-4965-afd3-f4de043fa01a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:36.451667 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fc395-959f-4965-afd3-f4de043fa01a None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:36.453345 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 763/1503] 10.222.0.26 () {54 vars in 963 bytes} [Thu Sep 22 14:54:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:54:36.460133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7f456-4457-43cd-945e-f9a1ac420ceb None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:36.460416 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7f456-4457-43cd-945e-f9a1ac420ceb None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:36.460797 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7f456-4457-43cd-945e-f9a1ac420ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:36.493728 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:36.540556 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-44e7f456-4457-43cd-945e-f9a1ac420ceb None None] MFA Rules not processed for user `1d265d111b534944b7812cf067e317a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:54:36.541765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-44e7f456-4457-43cd-945e-f9a1ac420ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:36.620541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 741/1504] 10.222.0.26 () {58 vars in 1060 bytes} [Thu Sep 22 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:54:36.628772 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fc395-959f-4965-afd3-f4de043fa01a None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:36.630269 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-dc7fc395-959f-4965-afd3-f4de043fa01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:36.681354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:36.738174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fc395-959f-4965-afd3-f4de043fa01a None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:36.811744 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de844e74-adec-4766-8590-3b81422ecebd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:36.812351 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de844e74-adec-4766-8590-3b81422ecebd service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:36.812629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de844e74-adec-4766-8590-3b81422ecebd service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:36.812898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de844e74-adec-4766-8590-3b81422ecebd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:36.814493 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-de844e74-adec-4766-8590-3b81422ecebd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:36.917108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de844e74-adec-4766-8590-3b81422ecebd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:36.919032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de844e74-adec-4766-8590-3b81422ecebd service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:36.922554 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:36.938138 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 764/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:54:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:54:36.984116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988c438-576b-4318-8a95-41a46e10e9ca None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:36.984582 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988c438-576b-4318-8a95-41a46e10e9ca None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:36.984685 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988c438-576b-4318-8a95-41a46e10e9ca None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:36.985920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 742/1506] 10.222.0.26 () {52 vars in 830 bytes} [Thu Sep 22 14:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:54:36.989555 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de844e74-adec-4766-8590-3b81422ecebd service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:36.990801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-de844e74-adec-4766-8590-3b81422ecebd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:37.095956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de844e74-adec-4766-8590-3b81422ecebd service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:37.189055 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f9f62-8c86-4a74-bbec-576b3a45b127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:37.189724 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f9f62-8c86-4a74-bbec-576b3a45b127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:37.189863 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f9f62-8c86-4a74-bbec-576b3a45b127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:37.190132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f9f62-8c86-4a74-bbec-576b3a45b127 admin admin] PATH_INFO: `/v3/users/fe48b7f3711541ee94333b5458dab84f` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:37.202916 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f9f62-8c86-4a74-bbec-576b3a45b127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe48b7f3711541ee94333b5458dab84f)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:37.203944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: WARNING py.warnings [None req-7b9f9f62-8c86-4a74-bbec-576b3a45b127 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:54:37.203944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: warnings.warn(msg) Sep 22 14:54:37.203944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: Sep 22 14:54:37.205159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f9f62-8c86-4a74-bbec-576b3a45b127 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:37.214701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:37.271322 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-7b9f9f62-8c86-4a74-bbec-576b3a45b127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe48b7f3711541ee94333b5458dab84f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:37.280959 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-7b9f9f62-8c86-4a74-bbec-576b3a45b127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe48b7f3711541ee94333b5458dab84f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:37.281522 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-7b9f9f62-8c86-4a74-bbec-576b3a45b127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe48b7f3711541ee94333b5458dab84f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:37.305476 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-7b9f9f62-8c86-4a74-bbec-576b3a45b127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe48b7f3711541ee94333b5458dab84f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:37.305930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-7b9f9f62-8c86-4a74-bbec-576b3a45b127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe48b7f3711541ee94333b5458dab84f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:37.325361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 765/1507] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:54:36 2022] DELETE /identity/v3/users/fe48b7f3711541ee94333b5458dab84f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:54:37.329719 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b67c93-ff86-4434-969f-b0bd6fcdf5d7 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:37.330000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b67c93-ff86-4434-969f-b0bd6fcdf5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:37.330294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b67c93-ff86-4434-969f-b0bd6fcdf5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:37.388725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-23b67c93-ff86-4434-969f-b0bd6fcdf5d7 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:54:37.389968 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-23b67c93-ff86-4434-969f-b0bd6fcdf5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:37.464524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:37.478376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 743/1508] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:54:37.487939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f9f62-8c86-4a74-bbec-576b3a45b127 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:37.489158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7b9f9f62-8c86-4a74-bbec-576b3a45b127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:37.590041 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f9f62-8c86-4a74-bbec-576b3a45b127 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:37.646415 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:37.669294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e44c3f-b176-463b-933b-baba7dabe6ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:37.669898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e44c3f-b176-463b-933b-baba7dabe6ea service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:37.670175 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e44c3f-b176-463b-933b-baba7dabe6ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:37.670442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e44c3f-b176-463b-933b-baba7dabe6ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:37.672108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e1e44c3f-b176-463b-933b-baba7dabe6ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:37.781006 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e44c3f-b176-463b-933b-baba7dabe6ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:37.782443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e44c3f-b176-463b-933b-baba7dabe6ea service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:37.801593 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 766/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:54:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:54:38.147840 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b67c93-ff86-4434-969f-b0bd6fcdf5d7 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:54:38.149143 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-23b67c93-ff86-4434-969f-b0bd6fcdf5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:54:38.161941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b67c93-ff86-4434-969f-b0bd6fcdf5d7 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:54:38.258746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159e059-d3fc-40d7-9ba6-2cf695a4b1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:54:38.259409 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159e059-d3fc-40d7-9ba6-2cf695a4b1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:54:38.259690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159e059-d3fc-40d7-9ba6-2cf695a4b1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:54:38.259960 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159e059-d3fc-40d7-9ba6-2cf695a4b1d3 admin admin] PATH_INFO: `/v3/projects/71d456a9f86540dbba2fc5bc83d3e139` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:54:38.270642 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159e059-d3fc-40d7-9ba6-2cf695a4b1d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71d456a9f86540dbba2fc5bc83d3e139)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:54:38.271704 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: WARNING py.warnings [None req-4159e059-d3fc-40d7-9ba6-2cf695a4b1d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:54:38.271704 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: warnings.warn(msg) Sep 22 14:54:38.271704 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: Sep 22 14:54:38.272915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159e059-d3fc-40d7-9ba6-2cf695a4b1d3 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:54:38.333182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-4159e059-d3fc-40d7-9ba6-2cf695a4b1d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71d456a9f86540dbba2fc5bc83d3e139', 'request_id': 'req-4159e059-d3fc-40d7-9ba6-2cf695a4b1d3'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:38.347166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-4159e059-d3fc-40d7-9ba6-2cf695a4b1d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71d456a9f86540dbba2fc5bc83d3e139', 'request_id': 'req-4159e059-d3fc-40d7-9ba6-2cf695a4b1d3'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:38.353610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-4159e059-d3fc-40d7-9ba6-2cf695a4b1d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71d456a9f86540dbba2fc5bc83d3e139', 'request_id': 'req-4159e059-d3fc-40d7-9ba6-2cf695a4b1d3'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:38.379987 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-4159e059-d3fc-40d7-9ba6-2cf695a4b1d3 admin admin] The token cache is being invalidate because project 71d456a9f86540dbba2fc5bc83d3e139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:54:38.380279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-4159e059-d3fc-40d7-9ba6-2cf695a4b1d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:38.380718 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-4159e059-d3fc-40d7-9ba6-2cf695a4b1d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:54:38.382151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 744/1510] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:54:38 2022] DELETE /identity/v3/projects/71d456a9f86540dbba2fc5bc83d3e139 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:54:47.798799 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=62 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:54:48.377794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:56:04.699596 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5a212-7da6-4f6f-80c8-8441e4cc1683 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:04.700174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5a212-7da6-4f6f-80c8-8441e4cc1683 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:04.700174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5a212-7da6-4f6f-80c8-8441e4cc1683 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:04.701388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 767/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:56:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:04.706609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdc66e-3f43-437c-8391-963ffd5c7d3e None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:04.707110 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdc66e-3f43-437c-8391-963ffd5c7d3e None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:04.707229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdc66e-3f43-437c-8391-963ffd5c7d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:04.801619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-97cdc66e-3f43-437c-8391-963ffd5c7d3e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:04.802885 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-97cdc66e-3f43-437c-8391-963ffd5c7d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:04.908009 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 745/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:56:05.069561 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af5a212-7da6-4f6f-80c8-8441e4cc1683 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:05.071736 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-2af5a212-7da6-4f6f-80c8-8441e4cc1683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:05.195873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af5a212-7da6-4f6f-80c8-8441e4cc1683 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:05.201772 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1ae10-74b5-4f23-9cfb-4cd5c669c4dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:05.202392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1ae10-74b5-4f23-9cfb-4cd5c669c4dc service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:05.202685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1ae10-74b5-4f23-9cfb-4cd5c669c4dc service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:05.202989 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1ae10-74b5-4f23-9cfb-4cd5c669c4dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:05.212187 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1ae10-74b5-4f23-9cfb-4cd5c669c4dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:05.213876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1ae10-74b5-4f23-9cfb-4cd5c669c4dc service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:05.223798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 768/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:56:06.313374 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c302ba-d11b-478e-901a-f2f5fd5a16db None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:06.313878 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c302ba-d11b-478e-901a-f2f5fd5a16db None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:06.313878 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c302ba-d11b-478e-901a-f2f5fd5a16db None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:06.315228 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 746/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:06.321590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1430b74-750d-43d0-95a5-991bb2faf125 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:06.322065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1430b74-750d-43d0-95a5-991bb2faf125 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:06.322174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1430b74-750d-43d0-95a5-991bb2faf125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:06.342540 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-a1430b74-750d-43d0-95a5-991bb2faf125 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:06.343756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a1430b74-750d-43d0-95a5-991bb2faf125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:06.351502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 769/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:56:06.452870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070acfc-9288-49e4-897a-3937d3bae92d None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:06.453168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070acfc-9288-49e4-897a-3937d3bae92d None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:06.453418 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070acfc-9288-49e4-897a-3937d3bae92d None None] PATH_INFO: `/v3` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:06.454691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 747/1516] 10.222.0.67 () {54 vars in 925 bytes} [Thu Sep 22 14:56:06 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:56:06.461204 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948e648-91bb-449d-ba91-b7299214718f None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:06.461579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948e648-91bb-449d-ba91-b7299214718f None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:06.461921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948e648-91bb-449d-ba91-b7299214718f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:06.507160 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070acfc-9288-49e4-897a-3937d3bae92d None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:06.523807 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070acfc-9288-49e4-897a-3937d3bae92d None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:06.528010 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-7948e648-91bb-449d-ba91-b7299214718f None None] MFA Rules not processed for user `47b6b34b4ee5492fb1f0699716e5118b`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:06.529215 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7948e648-91bb-449d-ba91-b7299214718f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:06.531655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d0599-6d84-404f-99ea-dd8d6eb04a84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:06.532298 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d0599-6d84-404f-99ea-dd8d6eb04a84 service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:06.532580 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d0599-6d84-404f-99ea-dd8d6eb04a84 service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:06.532854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d0599-6d84-404f-99ea-dd8d6eb04a84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:06.541988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d0599-6d84-404f-99ea-dd8d6eb04a84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:06.544103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d0599-6d84-404f-99ea-dd8d6eb04a84 service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:06.553748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 748/1517] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:56:06.585667 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:56:06.608215 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 770/1518] 10.222.0.67 () {58 vars in 1015 bytes} [Thu Sep 22 14:56:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:56:10.206174 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe213279-71e6-477e-8a9e-99419baed972 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:10.206642 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe213279-71e6-477e-8a9e-99419baed972 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:10.206761 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe213279-71e6-477e-8a9e-99419baed972 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:10.208107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 749/1519] 10.222.0.26 () {54 vars in 961 bytes} [Thu Sep 22 14:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:10.213196 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299d717-101e-456e-bdb8-71a3494e076c None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:10.213620 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299d717-101e-456e-bdb8-71a3494e076c None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:10.214234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299d717-101e-456e-bdb8-71a3494e076c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:10.238629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-6299d717-101e-456e-bdb8-71a3494e076c None None] MFA Rules not processed for user `47b6b34b4ee5492fb1f0699716e5118b`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:10.239871 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-6299d717-101e-456e-bdb8-71a3494e076c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:10.248457 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 771/1520] 10.222.0.26 () {58 vars in 1058 bytes} [Thu Sep 22 14:56:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:56:10.256530 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe213279-71e6-477e-8a9e-99419baed972 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:10.271122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe213279-71e6-477e-8a9e-99419baed972 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:10.276116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd725752-7d86-4319-8579-5a99d5610e07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b6b34b4ee5492fb1f0699716e5118b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:10.277197 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd725752-7d86-4319-8579-5a99d5610e07 service placement] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:10.277476 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd725752-7d86-4319-8579-5a99d5610e07 service placement] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:10.277745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd725752-7d86-4319-8579-5a99d5610e07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:10.285865 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd725752-7d86-4319-8579-5a99d5610e07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:10.287475 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd725752-7d86-4319-8579-5a99d5610e07 service placement] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:10.297585 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 750/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:56:18.055396 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3324e5-4da1-45fc-847e-ed3c5524326f None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:18.055941 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3324e5-4da1-45fc-847e-ed3c5524326f None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:18.056053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3324e5-4da1-45fc-847e-ed3c5524326f None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:18.057320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 772/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:18.063803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda0747-7c46-4af2-bd13-1d1fb72b3081 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:18.064287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda0747-7c46-4af2-bd13-1d1fb72b3081 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:18.064393 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda0747-7c46-4af2-bd13-1d1fb72b3081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:18.084426 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-4fda0747-7c46-4af2-bd13-1d1fb72b3081 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:18.085616 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4fda0747-7c46-4af2-bd13-1d1fb72b3081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:18.094332 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 751/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:56:18.248362 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3324e5-4da1-45fc-847e-ed3c5524326f None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:18.263720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3324e5-4da1-45fc-847e-ed3c5524326f None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:18.269175 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb275ac-391e-4b6e-9fc0-a6d534af4b25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:18.269777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb275ac-391e-4b6e-9fc0-a6d534af4b25 service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:18.270057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb275ac-391e-4b6e-9fc0-a6d534af4b25 service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:18.270330 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb275ac-391e-4b6e-9fc0-a6d534af4b25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:18.278430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb275ac-391e-4b6e-9fc0-a6d534af4b25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:18.280037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb275ac-391e-4b6e-9fc0-a6d534af4b25 service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:18.289332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 773/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:56:19.371677 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe4e96-8eae-412f-838e-8192acee3928 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:19.372168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe4e96-8eae-412f-838e-8192acee3928 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:19.372287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe4e96-8eae-412f-838e-8192acee3928 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:19.373545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 752/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 14:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:19.379992 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef135f-bfc5-47ca-aff7-8a5aeb01535d None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:19.380442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef135f-bfc5-47ca-aff7-8a5aeb01535d None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:19.380553 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef135f-bfc5-47ca-aff7-8a5aeb01535d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:19.400039 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-21ef135f-bfc5-47ca-aff7-8a5aeb01535d None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:19.401241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-21ef135f-bfc5-47ca-aff7-8a5aeb01535d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:19.409660 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 774/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 14:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:56:19.561939 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe4e96-8eae-412f-838e-8192acee3928 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:19.576864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe4e96-8eae-412f-838e-8192acee3928 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:19.582437 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb849552-00cd-48f5-9eb3-b3e0e28c86a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:19.583007 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb849552-00cd-48f5-9eb3-b3e0e28c86a7 service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:19.583286 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb849552-00cd-48f5-9eb3-b3e0e28c86a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:19.583555 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb849552-00cd-48f5-9eb3-b3e0e28c86a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:19.592520 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb849552-00cd-48f5-9eb3-b3e0e28c86a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:19.594097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb849552-00cd-48f5-9eb3-b3e0e28c86a7 service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:19.603545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 753/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:56:23.733458 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba203e57-25fb-4738-832e-4c90d2251956 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:23.733999 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba203e57-25fb-4738-832e-4c90d2251956 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:23.733999 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba203e57-25fb-4738-832e-4c90d2251956 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:23.735291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 775/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:23.740642 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bcf35-1a7f-4565-b631-c7677ef9c532 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:23.741091 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bcf35-1a7f-4565-b631-c7677ef9c532 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:23.741199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bcf35-1a7f-4565-b631-c7677ef9c532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:23.760981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-625bcf35-1a7f-4565-b631-c7677ef9c532 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:23.762821 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-625bcf35-1a7f-4565-b631-c7677ef9c532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:23.770705 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 754/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:56:24.063070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba203e57-25fb-4738-832e-4c90d2251956 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:24.064787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ba203e57-25fb-4738-832e-4c90d2251956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:24.141674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba203e57-25fb-4738-832e-4c90d2251956 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:24.145693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d516ab7-60a7-4b76-bb3f-6fa954dc6437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:24.146291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d516ab7-60a7-4b76-bb3f-6fa954dc6437 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:24.146564 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d516ab7-60a7-4b76-bb3f-6fa954dc6437 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:24.146861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d516ab7-60a7-4b76-bb3f-6fa954dc6437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:24.156237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d516ab7-60a7-4b76-bb3f-6fa954dc6437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:24.157655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d516ab7-60a7-4b76-bb3f-6fa954dc6437 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:24.167455 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 776/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:56:26.343746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b27d2d-9172-4bda-b889-b9d8d8e135d5 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:26.344243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b27d2d-9172-4bda-b889-b9d8d8e135d5 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:26.344358 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b27d2d-9172-4bda-b889-b9d8d8e135d5 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:26.345660 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 755/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:26.351633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7bbf1-9685-4b67-a457-04907c28cc7a None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:26.352092 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7bbf1-9685-4b67-a457-04907c28cc7a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:26.352197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7bbf1-9685-4b67-a457-04907c28cc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:26.372305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-00b7bbf1-9685-4b67-a457-04907c28cc7a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:26.373516 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-00b7bbf1-9685-4b67-a457-04907c28cc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:26.381443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 777/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:56:26.679353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b27d2d-9172-4bda-b889-b9d8d8e135d5 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:26.681066 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-59b27d2d-9172-4bda-b889-b9d8d8e135d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:26.694453 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b27d2d-9172-4bda-b889-b9d8d8e135d5 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:26.698093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cddaef-8e6d-4fa6-9924-28dd568a4551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:26.699342 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cddaef-8e6d-4fa6-9924-28dd568a4551 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:26.699625 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cddaef-8e6d-4fa6-9924-28dd568a4551 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:26.699899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cddaef-8e6d-4fa6-9924-28dd568a4551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:26.708743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cddaef-8e6d-4fa6-9924-28dd568a4551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:26.710165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cddaef-8e6d-4fa6-9924-28dd568a4551 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:26.719850 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 756/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:56:30.539257 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a648e-f66d-4de4-b4ae-b429f002075b None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:30.539797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a648e-f66d-4de4-b4ae-b429f002075b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:30.539797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a648e-f66d-4de4-b4ae-b429f002075b None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:30.542246 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 778/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:56:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:30.547936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b656c61-d4d4-4ef5-87b3-ac15f698b67a None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:30.548362 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b656c61-d4d4-4ef5-87b3-ac15f698b67a None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:30.548515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b656c61-d4d4-4ef5-87b3-ac15f698b67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:30.556171 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:56:30.570161 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-4b656c61-d4d4-4ef5-87b3-ac15f698b67a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:30.571319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4b656c61-d4d4-4ef5-87b3-ac15f698b67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:30.581417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 757/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:56:30.889314 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a648e-f66d-4de4-b4ae-b429f002075b None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:30.891320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-5d0a648e-f66d-4de4-b4ae-b429f002075b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:30.967873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a648e-f66d-4de4-b4ae-b429f002075b None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:30.971425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d624fef-be62-4531-b1a7-322bb947fce8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:30.972035 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d624fef-be62-4531-b1a7-322bb947fce8 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:30.972309 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d624fef-be62-4531-b1a7-322bb947fce8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:30.972579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d624fef-be62-4531-b1a7-322bb947fce8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:30.982173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d624fef-be62-4531-b1a7-322bb947fce8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:30.984922 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d624fef-be62-4531-b1a7-322bb947fce8 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:30.997709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 779/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:56:33.178130 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561efd1b-b3b7-40c9-bfbb-b0eec228ea40 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:33.178765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561efd1b-b3b7-40c9-bfbb-b0eec228ea40 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:33.178765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561efd1b-b3b7-40c9-bfbb-b0eec228ea40 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:33.180046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 758/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 14:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:56:33.185348 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bb639-f8c3-4882-b20d-7e3c87543f90 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:33.185768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bb639-f8c3-4882-b20d-7e3c87543f90 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:33.185921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bb639-f8c3-4882-b20d-7e3c87543f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:33.205430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-803bb639-f8c3-4882-b20d-7e3c87543f90 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:56:33.206666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-803bb639-f8c3-4882-b20d-7e3c87543f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:33.215115 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 780/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 14:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:56:33.521178 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561efd1b-b3b7-40c9-bfbb-b0eec228ea40 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:56:33.522868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-561efd1b-b3b7-40c9-bfbb-b0eec228ea40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:56:33.536146 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561efd1b-b3b7-40c9-bfbb-b0eec228ea40 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:56:33.539633 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d602cf-1844-48f6-8399-fb0e97880f9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:56:33.540213 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d602cf-1844-48f6-8399-fb0e97880f9b service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:56:33.540484 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d602cf-1844-48f6-8399-fb0e97880f9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:56:33.540748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d602cf-1844-48f6-8399-fb0e97880f9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:56:33.549453 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d602cf-1844-48f6-8399-fb0e97880f9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:56:33.550896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d602cf-1844-48f6-8399-fb0e97880f9b service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:56:33.560170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 759/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:56:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:56:43.200528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=92 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:56:43.556063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=8 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:57:57.944917 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03785ada-f973-4850-ad4f-0416af27de51 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:57.946145 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03785ada-f973-4850-ad4f-0416af27de51 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:57.946539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03785ada-f973-4850-ad4f-0416af27de51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:57.973434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-03785ada-f973-4850-ad4f-0416af27de51 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:57.974712 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-03785ada-f973-4850-ad4f-0416af27de51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:57.986028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 781/1540] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:57:57.995423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664906b4-75a1-4cad-a3d0-b974aa008626 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:57.996485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664906b4-75a1-4cad-a3d0-b974aa008626 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:57.996990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664906b4-75a1-4cad-a3d0-b974aa008626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.026173 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-664906b4-75a1-4cad-a3d0-b974aa008626 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:58.027811 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-664906b4-75a1-4cad-a3d0-b974aa008626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:58.041991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 760/1541] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:57:58.047411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03785ada-f973-4850-ad4f-0416af27de51 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:58.060728 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03785ada-f973-4850-ad4f-0416af27de51 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:58.071633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369a203-6504-46ff-9464-744610a8eefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:58.072261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369a203-6504-46ff-9464-744610a8eefa admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:58.072541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369a203-6504-46ff-9464-744610a8eefa admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:58.072815 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1369a203-6504-46ff-9464-744610a8eefa admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.073465 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369a203-6504-46ff-9464-744610a8eefa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:57:58.073768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369a203-6504-46ff-9464-744610a8eefa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:58.074946 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: WARNING py.warnings [None req-1369a203-6504-46ff-9464-744610a8eefa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:57:58.074946 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: warnings.warn(msg) Sep 22 14:57:58.074946 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: Sep 22 14:57:58.075828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1369a203-6504-46ff-9464-744610a8eefa admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:58.095102 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 782/1542] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:57:58.104324 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664906b4-75a1-4cad-a3d0-b974aa008626 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:58.120558 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664906b4-75a1-4cad-a3d0-b974aa008626 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:58.128352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d357a-b1fe-4a09-b028-9c9a62f2ab68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:58.129253 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d357a-b1fe-4a09-b028-9c9a62f2ab68 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:58.129765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d357a-b1fe-4a09-b028-9c9a62f2ab68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:58.130256 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d357a-b1fe-4a09-b028-9c9a62f2ab68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.131161 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d357a-b1fe-4a09-b028-9c9a62f2ab68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:58.132402 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: WARNING py.warnings [None req-e22d357a-b1fe-4a09-b028-9c9a62f2ab68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:57:58.132402 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: warnings.warn(msg) Sep 22 14:57:58.132402 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: Sep 22 14:57:58.134047 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d357a-b1fe-4a09-b028-9c9a62f2ab68 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:58.194896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 761/1543] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:57:58 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:57:58.200448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369a203-6504-46ff-9464-744610a8eefa admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:58.223615 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1369a203-6504-46ff-9464-744610a8eefa admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:58.316356 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e8ea5-3315-4952-9977-748e82645198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:58.316998 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e8ea5-3315-4952-9977-748e82645198 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:58.317278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e8ea5-3315-4952-9977-748e82645198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:58.317548 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e8ea5-3315-4952-9977-748e82645198 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.318201 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e8ea5-3315-4952-9977-748e82645198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:58.319288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: WARNING py.warnings [None req-4f0e8ea5-3315-4952-9977-748e82645198 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:57:58.319288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: warnings.warn(msg) Sep 22 14:57:58.319288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: Sep 22 14:57:58.320564 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e8ea5-3315-4952-9977-748e82645198 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:58.352369 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fdba3-8703-4181-bf36-b4e04500b0c7 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:58.353062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fdba3-8703-4181-bf36-b4e04500b0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:58.353564 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fdba3-8703-4181-bf36-b4e04500b0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.366827 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 783/1544] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:57:58 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:57:58.367968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e8ea5-3315-4952-9977-748e82645198 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:58.386584 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e8ea5-3315-4952-9977-748e82645198 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:58.392148 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-f69fdba3-8703-4181-bf36-b4e04500b0c7 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:58.393878 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f69fdba3-8703-4181-bf36-b4e04500b0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:58.401574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca348f-7753-4d28-919b-89a0b2071b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:58.402224 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca348f-7753-4d28-919b-89a0b2071b3c admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:58.402505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca348f-7753-4d28-919b-89a0b2071b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:58.402805 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca348f-7753-4d28-919b-89a0b2071b3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.403498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca348f-7753-4d28-919b-89a0b2071b3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:58.405318 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca348f-7753-4d28-919b-89a0b2071b3c admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:58.414401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 762/1545] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:57:58.422890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b72f9e-af04-4dea-85c3-bf83c27ef54a None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:58.423502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b72f9e-af04-4dea-85c3-bf83c27ef54a None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:58.424002 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b72f9e-af04-4dea-85c3-bf83c27ef54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.426869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 784/1546] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:57:58.432861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca348f-7753-4d28-919b-89a0b2071b3c admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:58.453684 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-71b72f9e-af04-4dea-85c3-bf83c27ef54a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:58.457139 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca348f-7753-4d28-919b-89a0b2071b3c admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:58.459047 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-71b72f9e-af04-4dea-85c3-bf83c27ef54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:58.466209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def6364-6826-45de-8d01-84e6b0df496c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:58.466877 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def6364-6826-45de-8d01-84e6b0df496c admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:58.467247 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def6364-6826-45de-8d01-84e6b0df496c admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:58.467527 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def6364-6826-45de-8d01-84e6b0df496c admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.468190 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def6364-6826-45de-8d01-84e6b0df496c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:58.469832 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def6364-6826-45de-8d01-84e6b0df496c admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:58.482268 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 763/1547] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:57:58.486684 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b72f9e-af04-4dea-85c3-bf83c27ef54a None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:58.493215 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 785/1548] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:57:58.496871 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def6364-6826-45de-8d01-84e6b0df496c admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:58.498429 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b72f9e-af04-4dea-85c3-bf83c27ef54a None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:58.509307 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff4700-b906-4cb4-854f-a46c4691b1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:58.509944 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff4700-b906-4cb4-854f-a46c4691b1eb admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:58.510219 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff4700-b906-4cb4-854f-a46c4691b1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:58.510484 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff4700-b906-4cb4-854f-a46c4691b1eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.511205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff4700-b906-4cb4-854f-a46c4691b1eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:57:58.511934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff4700-b906-4cb4-854f-a46c4691b1eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:58.513654 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff4700-b906-4cb4-854f-a46c4691b1eb admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:58.516143 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def6364-6826-45de-8d01-84e6b0df496c admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:58.532773 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04154e-8eda-4abf-82fb-2867b2b14956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:58.533466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04154e-8eda-4abf-82fb-2867b2b14956 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:58.533951 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04154e-8eda-4abf-82fb-2867b2b14956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:58.534155 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 764/1549] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:57:58.534590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04154e-8eda-4abf-82fb-2867b2b14956 admin admin] PATH_INFO: `/v3/projects/5801efe6a126482098daa0760452c327/users/9e0356a955d5491492307a423b5e43d6/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.542431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff4700-b906-4cb4-854f-a46c4691b1eb admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:58.559752 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04154e-8eda-4abf-82fb-2867b2b14956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5801efe6a126482098daa0760452c327, user_id=9e0356a955d5491492307a423b5e43d6, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:58.564004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04154e-8eda-4abf-82fb-2867b2b14956 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:58.566212 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff4700-b906-4cb4-854f-a46c4691b1eb admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:58.584712 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ca460-4409-4f06-96bd-d6488ac94a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:58.585335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ca460-4409-4f06-96bd-d6488ac94a78 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:58.585611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ca460-4409-4f06-96bd-d6488ac94a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:58.585879 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ca460-4409-4f06-96bd-d6488ac94a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.587100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ca460-4409-4f06-96bd-d6488ac94a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:58.589024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ca460-4409-4f06-96bd-d6488ac94a78 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:58.595562 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 786/1550] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:57:58 2022] PUT /identity/v3/projects/5801efe6a126482098daa0760452c327/users/9e0356a955d5491492307a423b5e43d6/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:57:58.608147 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74287ddd-bf6b-4d7f-8076-17b74ff76361 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:58.609018 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74287ddd-bf6b-4d7f-8076-17b74ff76361 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:58.609558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74287ddd-bf6b-4d7f-8076-17b74ff76361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.636028 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 765/1551] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:57:58 2022] POST /identity/v3/projects => generated 403 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:57:58.640145 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ca460-4409-4f06-96bd-d6488ac94a78 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:58.665629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-74287ddd-bf6b-4d7f-8076-17b74ff76361 None None] MFA Rules not processed for user `9e0356a955d5491492307a423b5e43d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:58.667845 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-74287ddd-bf6b-4d7f-8076-17b74ff76361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:58.775670 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ca460-4409-4f06-96bd-d6488ac94a78 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:58.782300 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 787/1552] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:57:58.784610 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916dbf9-482e-45da-a22c-e92be2b50747 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:58.785592 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916dbf9-482e-45da-a22c-e92be2b50747 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:58.786416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916dbf9-482e-45da-a22c-e92be2b50747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.788996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ed020-3ceb-4f6f-90be-8113b108f35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:58.792061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ed020-3ceb-4f6f-90be-8113b108f35d admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:58.792588 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ed020-3ceb-4f6f-90be-8113b108f35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:58.793081 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ed020-3ceb-4f6f-90be-8113b108f35d admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.793950 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ed020-3ceb-4f6f-90be-8113b108f35d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:58.796736 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ed020-3ceb-4f6f-90be-8113b108f35d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:58.836279 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-4916dbf9-482e-45da-a22c-e92be2b50747 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:58.837601 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-4916dbf9-482e-45da-a22c-e92be2b50747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:58.838570 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 766/1553] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:57:58 2022] POST /identity/v3/users => generated 302 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:57:58.840203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338e12f-916e-43b4-a756-b3daa30dd08e None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:58.841002 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338e12f-916e-43b4-a756-b3daa30dd08e None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:58.841492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338e12f-916e-43b4-a756-b3daa30dd08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.847155 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 788/1554] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:57:58.848056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916dbf9-482e-45da-a22c-e92be2b50747 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:58.859861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916dbf9-482e-45da-a22c-e92be2b50747 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:58.873177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d9bc2-216d-4f6a-899b-8bebb03b440a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:58.873811 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d9bc2-216d-4f6a-899b-8bebb03b440a admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:58.873921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-7338e12f-916e-43b4-a756-b3daa30dd08e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:58.874101 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d9bc2-216d-4f6a-899b-8bebb03b440a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:58.874357 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d9bc2-216d-4f6a-899b-8bebb03b440a admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.875066 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7338e12f-916e-43b4-a756-b3daa30dd08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:58.875176 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d9bc2-216d-4f6a-899b-8bebb03b440a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:58.877570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d9bc2-216d-4f6a-899b-8bebb03b440a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:58.882533 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:57:58.883858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 767/1555] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:57:58.887301 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2c164-53c1-436b-a373-264195c26f45 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:58.887588 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2c164-53c1-436b-a373-264195c26f45 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:58.887866 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2c164-53c1-436b-a373-264195c26f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.910496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-55b2c164-53c1-436b-a373-264195c26f45 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:58.911724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-55b2c164-53c1-436b-a373-264195c26f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:58.912701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 789/1556] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:57:58 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:57:58.914308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a7798-1c4c-4826-a5ba-39845b03dfc3 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:58.914594 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a7798-1c4c-4826-a5ba-39845b03dfc3 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:58.914944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a7798-1c4c-4826-a5ba-39845b03dfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:58.937432 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-7f3a7798-1c4c-4826-a5ba-39845b03dfc3 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:58.939120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7f3a7798-1c4c-4826-a5ba-39845b03dfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:58.972731 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:57:58.988040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 768/1557] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:57:58.988040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2c164-53c1-436b-a373-264195c26f45 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.007022 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2c164-53c1-436b-a373-264195c26f45 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.010448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 790/1558] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:57:59.011738 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b0b1c-1657-4e18-8d88-8660ed93e5df None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.012035 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b0b1c-1657-4e18-8d88-8660ed93e5df None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.012309 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b0b1c-1657-4e18-8d88-8660ed93e5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.018417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e88dd9-2b92-4de2-b908-2f8e77eea91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.019091 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e88dd9-2b92-4de2-b908-2f8e77eea91b admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.019352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e88dd9-2b92-4de2-b908-2f8e77eea91b admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.019704 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e88dd9-2b92-4de2-b908-2f8e77eea91b admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.020277 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e88dd9-2b92-4de2-b908-2f8e77eea91b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.021911 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e88dd9-2b92-4de2-b908-2f8e77eea91b admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.032305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-b81b0b1c-1657-4e18-8d88-8660ed93e5df None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:59.032850 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 769/1559] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:57:59.033471 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b81b0b1c-1657-4e18-8d88-8660ed93e5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:59.034440 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4b3e1-cf7f-4bd6-a9bd-29e82b86a274 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.034768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4b3e1-cf7f-4bd6-a9bd-29e82b86a274 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.035056 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4b3e1-cf7f-4bd6-a9bd-29e82b86a274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.042447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 791/1560] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:57:59.043711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab13ee-0e03-422f-a828-d88c3f355a2b None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.043999 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab13ee-0e03-422f-a828-d88c3f355a2b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.044280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab13ee-0e03-422f-a828-d88c3f355a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.056550 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-e3d4b3e1-cf7f-4bd6-a9bd-29e82b86a274 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:59.057754 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e3d4b3e1-cf7f-4bd6-a9bd-29e82b86a274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:59.063809 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-66ab13ee-0e03-422f-a828-d88c3f355a2b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:59.064982 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-66ab13ee-0e03-422f-a828-d88c3f355a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:59.065655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 770/1561] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:57:59.066411 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4b3e1-cf7f-4bd6-a9bd-29e82b86a274 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.073021 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 792/1562] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:57:59.076263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad2c0e-521a-4463-8d50-f2d2ffc14695 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.076263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad2c0e-521a-4463-8d50-f2d2ffc14695 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.076263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad2c0e-521a-4463-8d50-f2d2ffc14695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.078904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4b3e1-cf7f-4bd6-a9bd-29e82b86a274 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.083771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3235d19-64d2-434d-bec9-3b12ae7669e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.084374 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3235d19-64d2-434d-bec9-3b12ae7669e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.084654 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3235d19-64d2-434d-bec9-3b12ae7669e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.084925 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3235d19-64d2-434d-bec9-3b12ae7669e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.085555 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3235d19-64d2-434d-bec9-3b12ae7669e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.087777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3235d19-64d2-434d-bec9-3b12ae7669e6 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.094584 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-d1ad2c0e-521a-4463-8d50-f2d2ffc14695 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:59.095835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d1ad2c0e-521a-4463-8d50-f2d2ffc14695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:59.105080 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 793/1563] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:57:59.106377 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0fe2a-afca-46c8-812c-d65d80f439d2 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.106663 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0fe2a-afca-46c8-812c-d65d80f439d2 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.106965 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0fe2a-afca-46c8-812c-d65d80f439d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.114468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 771/1564] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/users => generated 293 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:57:59.115845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49988ac-117f-441e-a847-829871fc1b8b None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.116132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49988ac-117f-441e-a847-829871fc1b8b None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.116408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49988ac-117f-441e-a847-829871fc1b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.127938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-6de0fe2a-afca-46c8-812c-d65d80f439d2 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:59.129708 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-6de0fe2a-afca-46c8-812c-d65d80f439d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:59.137751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 794/1565] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:57:59.137999 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-c49988ac-117f-441e-a847-829871fc1b8b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:59.138616 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0fe2a-afca-46c8-812c-d65d80f439d2 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.139188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c49988ac-117f-441e-a847-829871fc1b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:59.147744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 772/1566] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:57:59.148380 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0fe2a-afca-46c8-812c-d65d80f439d2 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.148726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49988ac-117f-441e-a847-829871fc1b8b None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.153169 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7294ca-2882-4c34-a671-f5cf950471eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.153990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7294ca-2882-4c34-a671-f5cf950471eb admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.154272 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7294ca-2882-4c34-a671-f5cf950471eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.154548 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7294ca-2882-4c34-a671-f5cf950471eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.155229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7294ca-2882-4c34-a671-f5cf950471eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.156849 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7294ca-2882-4c34-a671-f5cf950471eb admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.158619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49988ac-117f-441e-a847-829871fc1b8b None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.163258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf43dd7-37dc-4d03-8a72-831d9ab69f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.163979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf43dd7-37dc-4d03-8a72-831d9ab69f7b admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.164261 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf43dd7-37dc-4d03-8a72-831d9ab69f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.165070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf43dd7-37dc-4d03-8a72-831d9ab69f7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.165704 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf43dd7-37dc-4d03-8a72-831d9ab69f7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:57:59.166004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf43dd7-37dc-4d03-8a72-831d9ab69f7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.167611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf43dd7-37dc-4d03-8a72-831d9ab69f7b admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.167889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 795/1567] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:57:59.169248 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526bea2-e1dd-4ee6-afec-5e627578bf9a None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.169539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526bea2-e1dd-4ee6-afec-5e627578bf9a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.169816 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526bea2-e1dd-4ee6-afec-5e627578bf9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.189444 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 773/1568] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:57:59.189444 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf43dd7-37dc-4d03-8a72-831d9ab69f7b admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.191092 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-7526bea2-e1dd-4ee6-afec-5e627578bf9a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:59.192289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7526bea2-e1dd-4ee6-afec-5e627578bf9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:59.194782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf43dd7-37dc-4d03-8a72-831d9ab69f7b admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.199605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddd065-2983-43f1-ac4e-d7761c897476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.200238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddd065-2983-43f1-ac4e-d7761c897476 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.200358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 796/1569] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:57:59.201570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96dc4b0-fe11-46e5-a65d-6d3845cf9146 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.201850 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96dc4b0-fe11-46e5-a65d-6d3845cf9146 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.202124 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96dc4b0-fe11-46e5-a65d-6d3845cf9146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.204061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddd065-2983-43f1-ac4e-d7761c897476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.204354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddd065-2983-43f1-ac4e-d7761c897476 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.205010 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddd065-2983-43f1-ac4e-d7761c897476 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:57:59.205321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddd065-2983-43f1-ac4e-d7761c897476 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.206977 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddd065-2983-43f1-ac4e-d7761c897476 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.220146 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 774/1570] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:57:59.220350 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddd065-2983-43f1-ac4e-d7761c897476 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.221492 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-a96dc4b0-fe11-46e5-a65d-6d3845cf9146 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:59.222687 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a96dc4b0-fe11-46e5-a65d-6d3845cf9146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:59.230530 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddd065-2983-43f1-ac4e-d7761c897476 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.231477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 797/1571] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:57:59.232395 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96dc4b0-fe11-46e5-a65d-6d3845cf9146 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.235548 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a0a63-0b0f-46f3-8a78-d01531f6a07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.236146 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a0a63-0b0f-46f3-8a78-d01531f6a07a admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.236426 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a0a63-0b0f-46f3-8a78-d01531f6a07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.236698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a0a63-0b0f-46f3-8a78-d01531f6a07a admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.237335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a0a63-0b0f-46f3-8a78-d01531f6a07a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.239309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a0a63-0b0f-46f3-8a78-d01531f6a07a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.244873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96dc4b0-fe11-46e5-a65d-6d3845cf9146 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.250062 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad14cda-5086-4820-9fd5-c1563468b683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.253998 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad14cda-5086-4820-9fd5-c1563468b683 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.254169 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 775/1572] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:57:59.254278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad14cda-5086-4820-9fd5-c1563468b683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.254278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad14cda-5086-4820-9fd5-c1563468b683 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.254462 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a0a63-0b0f-46f3-8a78-d01531f6a07a admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.254666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad14cda-5086-4820-9fd5-c1563468b683 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:57:59.254666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad14cda-5086-4820-9fd5-c1563468b683 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.254666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad14cda-5086-4820-9fd5-c1563468b683 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.264597 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a0a63-0b0f-46f3-8a78-d01531f6a07a admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.272103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 798/1573] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:57:59.272103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad14cda-5086-4820-9fd5-c1563468b683 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.274708 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f4137-d50d-4e70-8b99-0f01e12ba62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.274708 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f4137-d50d-4e70-8b99-0f01e12ba62b admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.275019 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f4137-d50d-4e70-8b99-0f01e12ba62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.275259 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f4137-d50d-4e70-8b99-0f01e12ba62b admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.275908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f4137-d50d-4e70-8b99-0f01e12ba62b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:57:59.276206 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f4137-d50d-4e70-8b99-0f01e12ba62b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.277815 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f4137-d50d-4e70-8b99-0f01e12ba62b admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.279056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad14cda-5086-4820-9fd5-c1563468b683 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.283800 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d03b14-005d-4446-bcf1-ce12462396b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.284394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d03b14-005d-4446-bcf1-ce12462396b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.284679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d03b14-005d-4446-bcf1-ce12462396b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.285010 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d03b14-005d-4446-bcf1-ce12462396b7 admin admin] PATH_INFO: `/v3/projects/5ff35d1955d84cb3abef1c748996b844/users/0956e9529ad544f6a2ae8cdcbd56ab3d/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.290914 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 776/1574] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:57:59.292536 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f4137-d50d-4e70-8b99-0f01e12ba62b admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.301047 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d03b14-005d-4446-bcf1-ce12462396b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff35d1955d84cb3abef1c748996b844, user_id=0956e9529ad544f6a2ae8cdcbd56ab3d, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.303519 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f4137-d50d-4e70-8b99-0f01e12ba62b admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.303911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d03b14-005d-4446-bcf1-ce12462396b7 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.308791 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3a169-8b98-478a-b224-511029d84966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.309401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3a169-8b98-478a-b224-511029d84966 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.309679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3a169-8b98-478a-b224-511029d84966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.309947 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3a169-8b98-478a-b224-511029d84966 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.310575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3a169-8b98-478a-b224-511029d84966 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.312535 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3a169-8b98-478a-b224-511029d84966 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.323473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 799/1575] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:57:59 2022] PUT /identity/v3/projects/5ff35d1955d84cb3abef1c748996b844/users/0956e9529ad544f6a2ae8cdcbd56ab3d/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:57:59.323675 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d03b14-005d-4446-bcf1-ce12462396b7 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.345957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 777/1576] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/projects => generated 393 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:57:59.347962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3a169-8b98-478a-b224-511029d84966 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.386157 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:57:59.390285 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d03b14-005d-4446-bcf1-ce12462396b7 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.411875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3a169-8b98-478a-b224-511029d84966 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.417042 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd5831-dd7c-479e-824b-d56176e20daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.417651 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd5831-dd7c-479e-824b-d56176e20daf admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.417927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd5831-dd7c-479e-824b-d56176e20daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.418198 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd5831-dd7c-479e-824b-d56176e20daf admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.418895 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd5831-dd7c-479e-824b-d56176e20daf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.420805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd5831-dd7c-479e-824b-d56176e20daf admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.448591 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 778/1577] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:57:59.449554 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd5831-dd7c-479e-824b-d56176e20daf admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.452907 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87842144-1763-427b-8bbd-13c17f2461de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.453525 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87842144-1763-427b-8bbd-13c17f2461de admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.453806 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87842144-1763-427b-8bbd-13c17f2461de admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.454077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87842144-1763-427b-8bbd-13c17f2461de admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.454724 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87842144-1763-427b-8bbd-13c17f2461de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:57:59.455054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87842144-1763-427b-8bbd-13c17f2461de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.456642 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87842144-1763-427b-8bbd-13c17f2461de admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.475873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 800/1578] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:57:59.475873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87842144-1763-427b-8bbd-13c17f2461de admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.480312 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87842144-1763-427b-8bbd-13c17f2461de admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.490983 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:57:59.527750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd5831-dd7c-479e-824b-d56176e20daf admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.532538 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ba8f2-1b56-4f00-84f8-ad69932adc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.533330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ba8f2-1b56-4f00-84f8-ad69932adc79 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.533605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ba8f2-1b56-4f00-84f8-ad69932adc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.534253 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ba8f2-1b56-4f00-84f8-ad69932adc79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.534838 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ba8f2-1b56-4f00-84f8-ad69932adc79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:57:59.535141 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ba8f2-1b56-4f00-84f8-ad69932adc79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.536792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ba8f2-1b56-4f00-84f8-ad69932adc79 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.537701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c05a07-de1c-485c-9e36-44ca563a0418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.538358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c05a07-de1c-485c-9e36-44ca563a0418 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.538632 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c05a07-de1c-485c-9e36-44ca563a0418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.538929 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c05a07-de1c-485c-9e36-44ca563a0418 admin admin] PATH_INFO: `/v3/projects/3ee9b835747f43bb9c5b8392301bb4ce/users/768e8cf10fc7410e9b97d7366414b417/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.549138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 779/1579] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:57:59.550132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ba8f2-1b56-4f00-84f8-ad69932adc79 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.560855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c05a07-de1c-485c-9e36-44ca563a0418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee9b835747f43bb9c5b8392301bb4ce, user_id=768e8cf10fc7410e9b97d7366414b417, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.561827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ba8f2-1b56-4f00-84f8-ad69932adc79 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.563635 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c05a07-de1c-485c-9e36-44ca563a0418 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.566529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c631b7-065d-4c49-a75c-5f7b2e7519ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.567186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c631b7-065d-4c49-a75c-5f7b2e7519ac admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.567467 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c631b7-065d-4c49-a75c-5f7b2e7519ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.567778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c631b7-065d-4c49-a75c-5f7b2e7519ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.568604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c631b7-065d-4c49-a75c-5f7b2e7519ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.570940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c631b7-065d-4c49-a75c-5f7b2e7519ac admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.577199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 801/1580] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:57:59 2022] PUT /identity/v3/projects/3ee9b835747f43bb9c5b8392301bb4ce/users/768e8cf10fc7410e9b97d7366414b417/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:57:59.578002 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c05a07-de1c-485c-9e36-44ca563a0418 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.600454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 780/1581] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:57:59.601784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1239d1b-055e-449f-af8a-d4c58b86f3ce None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.602646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1239d1b-055e-449f-af8a-d4c58b86f3ce None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.602924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1239d1b-055e-449f-af8a-d4c58b86f3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.621764 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-a1239d1b-055e-449f-af8a-d4c58b86f3ce None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:57:59.623013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a1239d1b-055e-449f-af8a-d4c58b86f3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:57:59.643026 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c05a07-de1c-485c-9e36-44ca563a0418 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.669613 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:57:59.684224 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 781/1582] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:57:59.685057 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1239d1b-055e-449f-af8a-d4c58b86f3ce None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.697391 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1239d1b-055e-449f-af8a-d4c58b86f3ce None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.701940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a4cb9-df55-4997-b292-4b4d8d2da564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.702533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a4cb9-df55-4997-b292-4b4d8d2da564 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.702856 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a4cb9-df55-4997-b292-4b4d8d2da564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.703016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da64868-f326-4366-b6a5-742f1a1c9918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.703186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a4cb9-df55-4997-b292-4b4d8d2da564 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.703567 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da64868-f326-4366-b6a5-742f1a1c9918 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.703721 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a4cb9-df55-4997-b292-4b4d8d2da564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.703871 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da64868-f326-4366-b6a5-742f1a1c9918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.704109 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da64868-f326-4366-b6a5-742f1a1c9918 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.704746 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da64868-f326-4366-b6a5-742f1a1c9918 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.705618 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a4cb9-df55-4997-b292-4b4d8d2da564 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.706627 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da64868-f326-4366-b6a5-742f1a1c9918 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.729504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 782/1583] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/users => generated 297 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:57:59.730583 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a4cb9-df55-4997-b292-4b4d8d2da564 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.738147 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 802/1584] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:57:59.739028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da64868-f326-4366-b6a5-742f1a1c9918 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.743846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a4cb9-df55-4997-b292-4b4d8d2da564 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.802083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a46374-4ef2-4ff6-b66b-5ced6aa0447b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.802623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a46374-4ef2-4ff6-b66b-5ced6aa0447b admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.802933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a46374-4ef2-4ff6-b66b-5ced6aa0447b admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.803209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a46374-4ef2-4ff6-b66b-5ced6aa0447b admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.803846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a46374-4ef2-4ff6-b66b-5ced6aa0447b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.804771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da64868-f326-4366-b6a5-742f1a1c9918 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.805909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a46374-4ef2-4ff6-b66b-5ced6aa0447b admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.809728 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c49a95-2c4b-4ddf-9472-96ccfe3ed046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.810079 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c49a95-2c4b-4ddf-9472-96ccfe3ed046 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.810357 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c49a95-2c4b-4ddf-9472-96ccfe3ed046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.810629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c49a95-2c4b-4ddf-9472-96ccfe3ed046 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.811284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c49a95-2c4b-4ddf-9472-96ccfe3ed046 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.813168 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c49a95-2c4b-4ddf-9472-96ccfe3ed046 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.827888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 783/1585] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:57:59.831602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a46374-4ef2-4ff6-b66b-5ced6aa0447b admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.842335 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 803/1586] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:57:59.842928 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a46374-4ef2-4ff6-b66b-5ced6aa0447b admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.846493 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c49a95-2c4b-4ddf-9472-96ccfe3ed046 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.852014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:57:59.905733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a920d85-4358-4661-91a1-3ea2f41c07eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.906343 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a920d85-4358-4661-91a1-3ea2f41c07eb admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.906627 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a920d85-4358-4661-91a1-3ea2f41c07eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.906927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a920d85-4358-4661-91a1-3ea2f41c07eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.907566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a920d85-4358-4661-91a1-3ea2f41c07eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.909070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c49a95-2c4b-4ddf-9472-96ccfe3ed046 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.909468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a920d85-4358-4661-91a1-3ea2f41c07eb admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.913646 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e52c01-ed83-4495-86c6-2e73bce24d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:57:59.914246 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e52c01-ed83-4495-86c6-2e73bce24d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:57:59.914687 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e52c01-ed83-4495-86c6-2e73bce24d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:57:59.915001 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e52c01-ed83-4495-86c6-2e73bce24d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:57:59.915642 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e52c01-ed83-4495-86c6-2e73bce24d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:57:59.917259 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e52c01-ed83-4495-86c6-2e73bce24d7a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:57:59.927247 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 804/1587] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:57:59.928105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e52c01-ed83-4495-86c6-2e73bce24d7a admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.937957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e52c01-ed83-4495-86c6-2e73bce24d7a admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:57:59.941995 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 784/1588] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:57:59.941995 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a920d85-4358-4661-91a1-3ea2f41c07eb admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:57:59.975715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:57:59.998074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a920d85-4358-4661-91a1-3ea2f41c07eb admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.000510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee5923-ca12-480e-840e-cc8cae876a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.001120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee5923-ca12-480e-840e-cc8cae876a5c admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.001393 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee5923-ca12-480e-840e-cc8cae876a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.001663 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee5923-ca12-480e-840e-cc8cae876a5c admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.001831 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23cd40-0656-4d27-a97a-0da72a9c1231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.002282 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee5923-ca12-480e-840e-cc8cae876a5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.002436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23cd40-0656-4d27-a97a-0da72a9c1231 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.002716 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23cd40-0656-4d27-a97a-0da72a9c1231 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.003030 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23cd40-0656-4d27-a97a-0da72a9c1231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.004214 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee5923-ca12-480e-840e-cc8cae876a5c admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.012714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23cd40-0656-4d27-a97a-0da72a9c1231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.014150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23cd40-0656-4d27-a97a-0da72a9c1231 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.026337 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 785/1589] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:57:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:00.027274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23cd40-0656-4d27-a97a-0da72a9c1231 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.031779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 805/1590] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:57:59 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:00.031779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee5923-ca12-480e-840e-cc8cae876a5c admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.036923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23cd40-0656-4d27-a97a-0da72a9c1231 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.042382 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e36c3-68c0-49f6-94db-28ad615cf7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.043008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e36c3-68c0-49f6-94db-28ad615cf7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.043287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e36c3-68c0-49f6-94db-28ad615cf7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.043554 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e36c3-68c0-49f6-94db-28ad615cf7c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.044189 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e36c3-68c0-49f6-94db-28ad615cf7c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.044683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee5923-ca12-480e-840e-cc8cae876a5c admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.045781 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e36c3-68c0-49f6-94db-28ad615cf7c1 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.049942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167b8b9-35e0-47ff-91bf-28f6cc56a01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.050925 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167b8b9-35e0-47ff-91bf-28f6cc56a01e admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.051408 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167b8b9-35e0-47ff-91bf-28f6cc56a01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.051754 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167b8b9-35e0-47ff-91bf-28f6cc56a01e admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.052536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167b8b9-35e0-47ff-91bf-28f6cc56a01e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.054635 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167b8b9-35e0-47ff-91bf-28f6cc56a01e admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.057483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 786/1591] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:00.058641 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e36c3-68c0-49f6-94db-28ad615cf7c1 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.070653 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e36c3-68c0-49f6-94db-28ad615cf7c1 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.077201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab55d0-9f30-4811-b458-feeab0e87537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.077594 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab55d0-9f30-4811-b458-feeab0e87537 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.078620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab55d0-9f30-4811-b458-feeab0e87537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.078780 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab55d0-9f30-4811-b458-feeab0e87537 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.079262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 806/1592] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:00 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:00.080111 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167b8b9-35e0-47ff-91bf-28f6cc56a01e admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.082887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab55d0-9f30-4811-b458-feeab0e87537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.084526 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab55d0-9f30-4811-b458-feeab0e87537 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.091588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167b8b9-35e0-47ff-91bf-28f6cc56a01e admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.096356 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6e359-930b-48e0-85e5-54335c68a791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.097039 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6e359-930b-48e0-85e5-54335c68a791 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.097160 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 787/1593] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:00.097725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6e359-930b-48e0-85e5-54335c68a791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.098170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab55d0-9f30-4811-b458-feeab0e87537 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.098303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6e359-930b-48e0-85e5-54335c68a791 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.099035 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6e359-930b-48e0-85e5-54335c68a791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.101480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6e359-930b-48e0-85e5-54335c68a791 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.109506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab55d0-9f30-4811-b458-feeab0e87537 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.115640 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65641e6-e5f8-4559-89b9-c4bbdfaad4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.116313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65641e6-e5f8-4559-89b9-c4bbdfaad4ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.116591 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65641e6-e5f8-4559-89b9-c4bbdfaad4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.116851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65641e6-e5f8-4559-89b9-c4bbdfaad4ed admin admin] PATH_INFO: `/v3/projects/3ee9b835747f43bb9c5b8392301bb4ce/users/768e8cf10fc7410e9b97d7366414b417/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.119469 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65641e6-e5f8-4559-89b9-c4bbdfaad4ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee9b835747f43bb9c5b8392301bb4ce, user_id=768e8cf10fc7410e9b97d7366414b417, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.120679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: WARNING py.warnings [None req-c65641e6-e5f8-4559-89b9-c4bbdfaad4ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:58:00.120679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: warnings.warn(msg) Sep 22 14:58:00.120679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: Sep 22 14:58:00.122592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65641e6-e5f8-4559-89b9-c4bbdfaad4ed admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.125847 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 807/1594] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:00 2022] POST /identity/v3/users => generated 294 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:00.129725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6e359-930b-48e0-85e5-54335c68a791 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.141121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6e359-930b-48e0-85e5-54335c68a791 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.142815 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 788/1595] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:00 2022] PUT /identity/v3/projects/3ee9b835747f43bb9c5b8392301bb4ce/users/768e8cf10fc7410e9b97d7366414b417/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:00.143820 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65641e6-e5f8-4559-89b9-c4bbdfaad4ed admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.212864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a6de4-0a3f-4f6b-8b5f-a5d208c04b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.213521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a6de4-0a3f-4f6b-8b5f-a5d208c04b25 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.213806 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a6de4-0a3f-4f6b-8b5f-a5d208c04b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.214078 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a6de4-0a3f-4f6b-8b5f-a5d208c04b25 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.214717 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a6de4-0a3f-4f6b-8b5f-a5d208c04b25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.216903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a6de4-0a3f-4f6b-8b5f-a5d208c04b25 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.219068 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65641e6-e5f8-4559-89b9-c4bbdfaad4ed admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.227393 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc997717-f0de-4d5e-aada-ec2016d1d5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.228818 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc997717-f0de-4d5e-aada-ec2016d1d5c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.228818 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc997717-f0de-4d5e-aada-ec2016d1d5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.228818 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc997717-f0de-4d5e-aada-ec2016d1d5c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.229539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc997717-f0de-4d5e-aada-ec2016d1d5c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.231271 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc997717-f0de-4d5e-aada-ec2016d1d5c5 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.244278 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 789/1596] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:00.245142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc997717-f0de-4d5e-aada-ec2016d1d5c5 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.250518 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:00.254204 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 808/1597] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:00 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:00.254204 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a6de4-0a3f-4f6b-8b5f-a5d208c04b25 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.256630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc997717-f0de-4d5e-aada-ec2016d1d5c5 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.263893 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6c3834-968e-48cf-a089-f09845704856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.264978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c3834-968e-48cf-a089-f09845704856 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.265651 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c3834-968e-48cf-a089-f09845704856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.265998 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a6de4-0a3f-4f6b-8b5f-a5d208c04b25 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.266234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c3834-968e-48cf-a089-f09845704856 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.266984 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6c3834-968e-48cf-a089-f09845704856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.268991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6c3834-968e-48cf-a089-f09845704856 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.271084 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0aa2ed-31d1-45e7-88a5-08f55d887a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.272132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aa2ed-31d1-45e7-88a5-08f55d887a3d admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.272489 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aa2ed-31d1-45e7-88a5-08f55d887a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.272835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aa2ed-31d1-45e7-88a5-08f55d887a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.273576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0aa2ed-31d1-45e7-88a5-08f55d887a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.275767 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0aa2ed-31d1-45e7-88a5-08f55d887a3d admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.285059 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 790/1598] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:00.285934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6c3834-968e-48cf-a089-f09845704856 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.292957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 809/1599] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:00.292957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0aa2ed-31d1-45e7-88a5-08f55d887a3d admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.297539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6c3834-968e-48cf-a089-f09845704856 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.302751 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daebf58-1213-46d2-b21a-01e9c4f3196a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.303528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daebf58-1213-46d2-b21a-01e9c4f3196a admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.303740 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daebf58-1213-46d2-b21a-01e9c4f3196a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.304083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daebf58-1213-46d2-b21a-01e9c4f3196a admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.304213 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0aa2ed-31d1-45e7-88a5-08f55d887a3d admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.306800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daebf58-1213-46d2-b21a-01e9c4f3196a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.309699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daebf58-1213-46d2-b21a-01e9c4f3196a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.310263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f1e5f-15a6-4cab-a63a-20ca886183fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.310989 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f1e5f-15a6-4cab-a63a-20ca886183fa admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.311444 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f1e5f-15a6-4cab-a63a-20ca886183fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.311565 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f1e5f-15a6-4cab-a63a-20ca886183fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.312457 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f1e5f-15a6-4cab-a63a-20ca886183fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.314770 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f1e5f-15a6-4cab-a63a-20ca886183fa admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.332110 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 791/1600] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:00.333541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dd83a-24f8-4afc-b42e-043a7525c920 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.333961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dd83a-24f8-4afc-b42e-043a7525c920 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.334338 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dd83a-24f8-4afc-b42e-043a7525c920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.343552 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 810/1601] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:00.343552 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f1e5f-15a6-4cab-a63a-20ca886183fa admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.360640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f1e5f-15a6-4cab-a63a-20ca886183fa admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.366048 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7302b1-03e1-4c69-a3d7-749b5a42496e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.366596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-b07dd83a-24f8-4afc-b42e-043a7525c920 None None] MFA Rules not processed for user `768e8cf10fc7410e9b97d7366414b417`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:00.367172 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7302b1-03e1-4c69-a3d7-749b5a42496e admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.368699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7302b1-03e1-4c69-a3d7-749b5a42496e admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.368836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b07dd83a-24f8-4afc-b42e-043a7525c920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:00.369012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7302b1-03e1-4c69-a3d7-749b5a42496e admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.369465 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7302b1-03e1-4c69-a3d7-749b5a42496e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.371205 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7302b1-03e1-4c69-a3d7-749b5a42496e admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.386876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 811/1602] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:00.386876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7302b1-03e1-4c69-a3d7-749b5a42496e admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.403345 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7302b1-03e1-4c69-a3d7-749b5a42496e admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.412432 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31b58f-1eec-4b0d-a867-5a36ecbefe42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.413136 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31b58f-1eec-4b0d-a867-5a36ecbefe42 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.413900 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31b58f-1eec-4b0d-a867-5a36ecbefe42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.414180 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31b58f-1eec-4b0d-a867-5a36ecbefe42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.414944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31b58f-1eec-4b0d-a867-5a36ecbefe42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.417004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31b58f-1eec-4b0d-a867-5a36ecbefe42 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.433740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 812/1603] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:00.434599 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31b58f-1eec-4b0d-a867-5a36ecbefe42 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.448849 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31b58f-1eec-4b0d-a867-5a36ecbefe42 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.455616 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d49abd-d41a-4ec5-8701-bf555cf1b5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.456200 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d49abd-d41a-4ec5-8701-bf555cf1b5bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.456479 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d49abd-d41a-4ec5-8701-bf555cf1b5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.456752 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d49abd-d41a-4ec5-8701-bf555cf1b5bc admin admin] PATH_INFO: `/v3/projects/6a15aa877b1b482fa70f65ecd4f8d1f3/users/038365d6ba7041f4af386767d98ed709/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.464019 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 792/1604] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:00.465178 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07dd83a-24f8-4afc-b42e-043a7525c920 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.480830 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d49abd-d41a-4ec5-8701-bf555cf1b5bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a15aa877b1b482fa70f65ecd4f8d1f3, user_id=038365d6ba7041f4af386767d98ed709, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.483608 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d49abd-d41a-4ec5-8701-bf555cf1b5bc admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.486023 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07dd83a-24f8-4afc-b42e-043a7525c920 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.491918 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331915fa-6628-4afb-a5d5-cc50e6a5356a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.492622 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331915fa-6628-4afb-a5d5-cc50e6a5356a admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.494207 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331915fa-6628-4afb-a5d5-cc50e6a5356a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.494207 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331915fa-6628-4afb-a5d5-cc50e6a5356a admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.494399 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331915fa-6628-4afb-a5d5-cc50e6a5356a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.496440 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331915fa-6628-4afb-a5d5-cc50e6a5356a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.501013 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 813/1605] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:00 2022] PUT /identity/v3/projects/6a15aa877b1b482fa70f65ecd4f8d1f3/users/038365d6ba7041f4af386767d98ed709/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:00.502011 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d49abd-d41a-4ec5-8701-bf555cf1b5bc admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.510707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 793/1606] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:00.514873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331915fa-6628-4afb-a5d5-cc50e6a5356a admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.518718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:00.526567 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331915fa-6628-4afb-a5d5-cc50e6a5356a admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.578604 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d49abd-d41a-4ec5-8701-bf555cf1b5bc admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.583434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169de5a-1d6f-4ec2-a9a6-ea886a654f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.584717 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169de5a-1d6f-4ec2-a9a6-ea886a654f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.585003 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169de5a-1d6f-4ec2-a9a6-ea886a654f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.585259 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169de5a-1d6f-4ec2-a9a6-ea886a654f0a admin admin] PATH_INFO: `/v3/projects/88d7db84f4364c589668c55826cce241/users/2c38e2a135634a0fb0fb735c61cabf59/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.591395 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602629e1-48ed-4161-839c-84c4024daee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.592394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602629e1-48ed-4161-839c-84c4024daee5 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.593075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602629e1-48ed-4161-839c-84c4024daee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.593500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602629e1-48ed-4161-839c-84c4024daee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.595469 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602629e1-48ed-4161-839c-84c4024daee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.597821 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602629e1-48ed-4161-839c-84c4024daee5 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.601017 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169de5a-1d6f-4ec2-a9a6-ea886a654f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=88d7db84f4364c589668c55826cce241, user_id=2c38e2a135634a0fb0fb735c61cabf59, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.604518 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6169de5a-1d6f-4ec2-a9a6-ea886a654f0a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.612457 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 794/1607] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:00.613760 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602629e1-48ed-4161-839c-84c4024daee5 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.626658 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602629e1-48ed-4161-839c-84c4024daee5 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.637367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 814/1608] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:00 2022] PUT /identity/v3/projects/88d7db84f4364c589668c55826cce241/users/2c38e2a135634a0fb0fb735c61cabf59/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:00.637367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169de5a-1d6f-4ec2-a9a6-ea886a654f0a admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.703296 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:00.710301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169de5a-1d6f-4ec2-a9a6-ea886a654f0a admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.715271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82144e-01d0-42ff-b8eb-620cb5794f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.716130 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82144e-01d0-42ff-b8eb-620cb5794f91 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.716403 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82144e-01d0-42ff-b8eb-620cb5794f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.716527 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c840cce-b917-4837-8b31-c6edd7216433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.716723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82144e-01d0-42ff-b8eb-620cb5794f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.717135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c840cce-b917-4837-8b31-c6edd7216433 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.717373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82144e-01d0-42ff-b8eb-620cb5794f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.717477 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c840cce-b917-4837-8b31-c6edd7216433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.717674 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c840cce-b917-4837-8b31-c6edd7216433 admin admin] PATH_INFO: `/v3/projects/dc8a4487033d4e4aa231cc5873def0cf/users/f298f0c6a9e44efba2a1e2fc648a0f5d/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.719074 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82144e-01d0-42ff-b8eb-620cb5794f91 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.731279 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 815/1609] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:00.735558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82144e-01d0-42ff-b8eb-620cb5794f91 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.760501 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c840cce-b917-4837-8b31-c6edd7216433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc8a4487033d4e4aa231cc5873def0cf, user_id=f298f0c6a9e44efba2a1e2fc648a0f5d, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.763919 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c840cce-b917-4837-8b31-c6edd7216433 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.786209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 795/1610] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:00 2022] PUT /identity/v3/projects/dc8a4487033d4e4aa231cc5873def0cf/users/f298f0c6a9e44efba2a1e2fc648a0f5d/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:00.786209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986acd5a-1838-4e54-b4ba-b440bf92a96d None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.786209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986acd5a-1838-4e54-b4ba-b440bf92a96d None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.786209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986acd5a-1838-4e54-b4ba-b440bf92a96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.797824 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82144e-01d0-42ff-b8eb-620cb5794f91 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.808502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-986acd5a-1838-4e54-b4ba-b440bf92a96d None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:00.809745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-986acd5a-1838-4e54-b4ba-b440bf92a96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:00.851867 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:00.862332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd9a88-85cb-41f9-868f-d60f18b7183c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.863776 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd9a88-85cb-41f9-868f-d60f18b7183c service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.864572 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd9a88-85cb-41f9-868f-d60f18b7183c service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.864733 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd9a88-85cb-41f9-868f-d60f18b7183c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.887771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 796/1611] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:00.887771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986acd5a-1838-4e54-b4ba-b440bf92a96d None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.898632 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986acd5a-1838-4e54-b4ba-b440bf92a96d None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:00.905305 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07866c-4edc-48c9-bf1a-cc429f4e5a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:00.906516 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07866c-4edc-48c9-bf1a-cc429f4e5a79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.906851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07866c-4edc-48c9-bf1a-cc429f4e5a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.907125 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07866c-4edc-48c9-bf1a-cc429f4e5a79 admin admin] PATH_INFO: `/v3/projects/d3cda271ef3244f391962778ed33c351/users/0addaa15e4c844a5acd84c3a5e3869ff/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.924833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07866c-4edc-48c9-bf1a-cc429f4e5a79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3cda271ef3244f391962778ed33c351, user_id=0addaa15e4c844a5acd84c3a5e3869ff, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.927715 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07866c-4edc-48c9-bf1a-cc429f4e5a79 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.935633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd9a88-85cb-41f9-868f-d60f18b7183c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:00.937358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd9a88-85cb-41f9-868f-d60f18b7183c service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:00.946808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 797/1612] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:00 2022] PUT /identity/v3/projects/d3cda271ef3244f391962778ed33c351/users/0addaa15e4c844a5acd84c3a5e3869ff/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:00.947675 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07866c-4edc-48c9-bf1a-cc429f4e5a79 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:00.951577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 816/1613] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:00.952848 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810763e-3ac3-4d08-ba45-950fc6034f65 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:00.953136 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810763e-3ac3-4d08-ba45-950fc6034f65 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:00.953409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810763e-3ac3-4d08-ba45-950fc6034f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:00.986001 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-8810763e-3ac3-4d08-ba45-950fc6034f65 None None] MFA Rules not processed for user `2c38e2a135634a0fb0fb735c61cabf59`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:00.987413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-8810763e-3ac3-4d08-ba45-950fc6034f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:01.016926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07866c-4edc-48c9-bf1a-cc429f4e5a79 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:01.021833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ff398-a55e-4fb9-9efa-7595fef229c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:01.022546 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ff398-a55e-4fb9-9efa-7595fef229c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:01.022842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ff398-a55e-4fb9-9efa-7595fef229c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:01.023180 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ff398-a55e-4fb9-9efa-7595fef229c2 admin admin] PATH_INFO: `/v3/projects/aef8a9d089c9446697837d5ebe2154c8/users/23d683cdc90041b5b29767fa2e2107e2/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:01.047837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ff398-a55e-4fb9-9efa-7595fef229c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aef8a9d089c9446697837d5ebe2154c8, user_id=23d683cdc90041b5b29767fa2e2107e2, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:01.051510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ff398-a55e-4fb9-9efa-7595fef229c2 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:01.066498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 817/1614] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:01.068186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8810763e-3ac3-4d08-ba45-950fc6034f65 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:01.070569 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 798/1615] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:00 2022] PUT /identity/v3/projects/aef8a9d089c9446697837d5ebe2154c8/users/23d683cdc90041b5b29767fa2e2107e2/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:01.078102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c993e-d0db-4492-a2b5-df39a5bfff67 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:01.078388 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c993e-d0db-4492-a2b5-df39a5bfff67 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:01.078661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c993e-d0db-4492-a2b5-df39a5bfff67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:01.081352 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8810763e-3ac3-4d08-ba45-950fc6034f65 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:01.117333 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-241c993e-d0db-4492-a2b5-df39a5bfff67 None None] MFA Rules not processed for user `0956e9529ad544f6a2ae8cdcbd56ab3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:01.118578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-241c993e-d0db-4492-a2b5-df39a5bfff67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:01.141662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:01.166764 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a55de4-dd57-43e2-9c5f-a0ae9dc55889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:01.167477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a55de4-dd57-43e2-9c5f-a0ae9dc55889 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:01.167762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a55de4-dd57-43e2-9c5f-a0ae9dc55889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:01.168043 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a55de4-dd57-43e2-9c5f-a0ae9dc55889 admin admin] PATH_INFO: `/v3/projects/1073a3c6ec9d4b96aaa7fc219b87343f/users/24d4b3517d1b40eb87714fd7f56a3df5/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:01.183941 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a55de4-dd57-43e2-9c5f-a0ae9dc55889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1073a3c6ec9d4b96aaa7fc219b87343f, user_id=24d4b3517d1b40eb87714fd7f56a3df5, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:01.186699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a55de4-dd57-43e2-9c5f-a0ae9dc55889 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:01.197995 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 799/1616] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:01 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:01.199604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffd16d-7582-43c3-bc7d-568b65ae3b89 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:01.199887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffd16d-7582-43c3-bc7d-568b65ae3b89 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:01.200161 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffd16d-7582-43c3-bc7d-568b65ae3b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:01.204900 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 818/1617] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:01 2022] PUT /identity/v3/projects/1073a3c6ec9d4b96aaa7fc219b87343f/users/24d4b3517d1b40eb87714fd7f56a3df5/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:01.205962 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a55de4-dd57-43e2-9c5f-a0ae9dc55889 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:01.226665 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:01.233835 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-f3ffd16d-7582-43c3-bc7d-568b65ae3b89 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:01.235207 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f3ffd16d-7582-43c3-bc7d-568b65ae3b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:01.288096 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a55de4-dd57-43e2-9c5f-a0ae9dc55889 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:01.291875 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84eb478-0e04-405f-8d66-01e5ad7a2c10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:01.292485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84eb478-0e04-405f-8d66-01e5ad7a2c10 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:01.292779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84eb478-0e04-405f-8d66-01e5ad7a2c10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:01.293040 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84eb478-0e04-405f-8d66-01e5ad7a2c10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:01.303681 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 800/1618] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:01.306327 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdaf5bb-5338-4fd1-bdcf-f7986014064f None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:01.306604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdaf5bb-5338-4fd1-bdcf-f7986014064f None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:01.306925 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdaf5bb-5338-4fd1-bdcf-f7986014064f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:01.335006 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-bfdaf5bb-5338-4fd1-bdcf-f7986014064f None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:01.337165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-bfdaf5bb-5338-4fd1-bdcf-f7986014064f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:01.351686 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 801/1619] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:01.363554 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5e332-e9fd-4b1b-8fff-a15ed1e42285 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:01.364574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5e332-e9fd-4b1b-8fff-a15ed1e42285 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:01.364928 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5e332-e9fd-4b1b-8fff-a15ed1e42285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:01.386175 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-9ef5e332-e9fd-4b1b-8fff-a15ed1e42285 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:01.387193 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84eb478-0e04-405f-8d66-01e5ad7a2c10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:01.387419 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-9ef5e332-e9fd-4b1b-8fff-a15ed1e42285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:01.388854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84eb478-0e04-405f-8d66-01e5ad7a2c10 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:01.396295 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 802/1620] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:01.400235 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 819/1621] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:01.400925 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84eb478-0e04-405f-8d66-01e5ad7a2c10 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:01.401417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f7e01-ff62-4ecd-8622-a3fec16cdec4 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:01.401642 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f7e01-ff62-4ecd-8622-a3fec16cdec4 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:01.401922 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f7e01-ff62-4ecd-8622-a3fec16cdec4 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:01.403275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 803/1622] 10.222.0.26 () {54 vars in 961 bytes} [Thu Sep 22 14:58:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:01.404704 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84565e4e-271b-460b-9888-345441155edb None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:01.405074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84565e4e-271b-460b-9888-345441155edb None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:01.405343 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84565e4e-271b-460b-9888-345441155edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:01.417654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84eb478-0e04-405f-8d66-01e5ad7a2c10 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:01.424023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b135c28-ce08-4b56-8b87-60ff75d53138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:01.424946 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b135c28-ce08-4b56-8b87-60ff75d53138 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:01.425466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b135c28-ce08-4b56-8b87-60ff75d53138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:01.425814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b135c28-ce08-4b56-8b87-60ff75d53138 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:01.426511 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b135c28-ce08-4b56-8b87-60ff75d53138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:01.428762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b135c28-ce08-4b56-8b87-60ff75d53138 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:01.437484 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-84565e4e-271b-460b-9888-345441155edb None None] MFA Rules not processed for user `24d4b3517d1b40eb87714fd7f56a3df5`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:01.439181 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-84565e4e-271b-460b-9888-345441155edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:01.482105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 820/1623] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:01 2022] POST /identity/v3/projects => generated 403 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:01.483476 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b135c28-ce08-4b56-8b87-60ff75d53138 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:01.526779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 804/1624] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:01.528604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84565e4e-271b-460b-9888-345441155edb None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:01.544071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84565e4e-271b-460b-9888-345441155edb None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:01.567060 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b135c28-ce08-4b56-8b87-60ff75d53138 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:01.572908 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984fdf2-3019-45e8-a147-1c1d14b14165 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:01.574059 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984fdf2-3019-45e8-a147-1c1d14b14165 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:01.574494 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984fdf2-3019-45e8-a147-1c1d14b14165 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:01.575077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984fdf2-3019-45e8-a147-1c1d14b14165 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:01.590083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:01.604672 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4c708-8223-4825-af95-5b2ab567fb77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:01.605430 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4c708-8223-4825-af95-5b2ab567fb77 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:01.605807 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4c708-8223-4825-af95-5b2ab567fb77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:01.606209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4c708-8223-4825-af95-5b2ab567fb77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:01.639438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:01.656578 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984fdf2-3019-45e8-a147-1c1d14b14165 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:01.658919 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984fdf2-3019-45e8-a147-1c1d14b14165 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:01.678968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 821/1625] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:01.681029 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f9a87-faf2-44a2-90ee-232c0ac13f9b None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:01.681340 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f9a87-faf2-44a2-90ee-232c0ac13f9b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:01.681586 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f9a87-faf2-44a2-90ee-232c0ac13f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:01.688053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4c708-8223-4825-af95-5b2ab567fb77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:01.691269 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4c708-8223-4825-af95-5b2ab567fb77 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:01.706492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 805/1626] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:01.707806 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4c708-8223-4825-af95-5b2ab567fb77 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:01.718968 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4c708-8223-4825-af95-5b2ab567fb77 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:01.723962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356f6ee-d898-4a2b-b8b6-92aae5722ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:01.724509 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356f6ee-d898-4a2b-b8b6-92aae5722ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:01.724974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356f6ee-d898-4a2b-b8b6-92aae5722ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:01.725326 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356f6ee-d898-4a2b-b8b6-92aae5722ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:01.738920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8356f6ee-d898-4a2b-b8b6-92aae5722ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:01.740735 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8356f6ee-d898-4a2b-b8b6-92aae5722ba9 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:01.742207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-026f9a87-faf2-44a2-90ee-232c0ac13f9b None None] MFA Rules not processed for user `0c9be41c39494e54a14398a1f03c4d69`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:01.743982 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-026f9a87-faf2-44a2-90ee-232c0ac13f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:01.756342 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 806/1627] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:01.757662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356f6ee-d898-4a2b-b8b6-92aae5722ba9 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:01.769635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356f6ee-d898-4a2b-b8b6-92aae5722ba9 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:01.775708 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a88d43-deb8-4234-ae6a-b9c1f27f10ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:01.776359 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a88d43-deb8-4234-ae6a-b9c1f27f10ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:01.776639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a88d43-deb8-4234-ae6a-b9c1f27f10ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:01.776927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a88d43-deb8-4234-ae6a-b9c1f27f10ec admin admin] PATH_INFO: `/v3/users/9e0356a955d5491492307a423b5e43d6` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:01.778217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a88d43-deb8-4234-ae6a-b9c1f27f10ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e0356a955d5491492307a423b5e43d6)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:01.779634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: WARNING py.warnings [None req-80a88d43-deb8-4234-ae6a-b9c1f27f10ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:58:01.779634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: warnings.warn(msg) Sep 22 14:58:01.779634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: Sep 22 14:58:01.780865 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a88d43-deb8-4234-ae6a-b9c1f27f10ec admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:01.871363 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 822/1628] 10.222.0.26 () {58 vars in 1058 bytes} [Thu Sep 22 14:58:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:01.873535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f9a87-faf2-44a2-90ee-232c0ac13f9b None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:01.884138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-80a88d43-deb8-4234-ae6a-b9c1f27f10ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e0356a955d5491492307a423b5e43d6'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:01.886659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f9a87-faf2-44a2-90ee-232c0ac13f9b None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:01.893525 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4c0c4-038d-4f13-bce5-acc7efefd1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:01.894529 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4c0c4-038d-4f13-bce5-acc7efefd1e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:01.894930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4c0c4-038d-4f13-bce5-acc7efefd1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:01.895259 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4c0c4-038d-4f13-bce5-acc7efefd1e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:01.896280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4c0c4-038d-4f13-bce5-acc7efefd1e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:01.897580 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-80a88d43-deb8-4234-ae6a-b9c1f27f10ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e0356a955d5491492307a423b5e43d6'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:01.899129 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4c0c4-038d-4f13-bce5-acc7efefd1e7 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:01.916258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-80a88d43-deb8-4234-ae6a-b9c1f27f10ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e0356a955d5491492307a423b5e43d6'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:01.933065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 823/1629] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:01 2022] POST /identity/v3/users => generated 308 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:01.933065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4c0c4-038d-4f13-bce5-acc7efefd1e7 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:01.951942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4c0c4-038d-4f13-bce5-acc7efefd1e7 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:01.952386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-80a88d43-deb8-4234-ae6a-b9c1f27f10ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e0356a955d5491492307a423b5e43d6'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:01.953329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-80a88d43-deb8-4234-ae6a-b9c1f27f10ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e0356a955d5491492307a423b5e43d6'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:01.959457 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cb82f-7c4d-44dd-8548-b6574f9cd88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:01.960221 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cb82f-7c4d-44dd-8548-b6574f9cd88a admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:01.960574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cb82f-7c4d-44dd-8548-b6574f9cd88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:01.961043 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cb82f-7c4d-44dd-8548-b6574f9cd88a admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:01.962124 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2cb82f-7c4d-44dd-8548-b6574f9cd88a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:01.964263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 807/1630] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:01 2022] DELETE /identity/v3/users/9e0356a955d5491492307a423b5e43d6 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:01.964994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2cb82f-7c4d-44dd-8548-b6574f9cd88a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:01.965808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a88d43-deb8-4234-ae6a-b9c1f27f10ec admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:01.967747 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-80a88d43-deb8-4234-ae6a-b9c1f27f10ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:02.011829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 824/1631] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:01 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:02.011829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cb82f-7c4d-44dd-8548-b6574f9cd88a admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:02.012769 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-dc2cb82f-7c4d-44dd-8548-b6574f9cd88a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:02.033100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:02.078321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a88d43-deb8-4234-ae6a-b9c1f27f10ec admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:02.145160 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cb82f-7c4d-44dd-8548-b6574f9cd88a admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:02.155502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fa49d-4457-47d6-aff7-ee57f47a0ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:02.156753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fa49d-4457-47d6-aff7-ee57f47a0ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:02.157214 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fa49d-4457-47d6-aff7-ee57f47a0ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:02.157648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fa49d-4457-47d6-aff7-ee57f47a0ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:02.158501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fa49d-4457-47d6-aff7-ee57f47a0ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:02.160410 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c37d77-9e2e-42c7-8c7b-64d00a45fc6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:02.160932 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c37d77-9e2e-42c7-8c7b-64d00a45fc6f service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:02.161186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fa49d-4457-47d6-aff7-ee57f47a0ee1 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:02.161300 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c37d77-9e2e-42c7-8c7b-64d00a45fc6f service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:02.161477 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c37d77-9e2e-42c7-8c7b-64d00a45fc6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:02.163395 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-95c37d77-9e2e-42c7-8c7b-64d00a45fc6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:02.196100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 825/1632] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:02.197796 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf371a-5569-4cda-92dc-b196d3957e74 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:02.198159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf371a-5569-4cda-92dc-b196d3957e74 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:02.198915 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf371a-5569-4cda-92dc-b196d3957e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:02.225953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-2adf371a-5569-4cda-92dc-b196d3957e74 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:02.228381 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-2adf371a-5569-4cda-92dc-b196d3957e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:02.257149 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 826/1633] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:02.257149 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf371a-5569-4cda-92dc-b196d3957e74 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:02.257590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-2adf371a-5569-4cda-92dc-b196d3957e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:02.269417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c37d77-9e2e-42c7-8c7b-64d00a45fc6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:02.272591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf371a-5569-4cda-92dc-b196d3957e74 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:02.275375 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c37d77-9e2e-42c7-8c7b-64d00a45fc6f service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:02.280908 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6b101-ae99-4f44-8709-548f4cce43d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:02.284387 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6b101-ae99-4f44-8709-548f4cce43d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:02.284852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6b101-ae99-4f44-8709-548f4cce43d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:02.285595 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6b101-ae99-4f44-8709-548f4cce43d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:02.286558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6b101-ae99-4f44-8709-548f4cce43d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:02.288719 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6b101-ae99-4f44-8709-548f4cce43d1 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:02.295884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 808/1634] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:02.296913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c37d77-9e2e-42c7-8c7b-64d00a45fc6f service nova] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:02.312910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:02.320075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c37d77-9e2e-42c7-8c7b-64d00a45fc6f service nova] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:02.326426 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3689fcc-ce2f-491b-8f15-b3ba1dfb100d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:02.331618 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3689fcc-ce2f-491b-8f15-b3ba1dfb100d admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:02.331896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3689fcc-ce2f-491b-8f15-b3ba1dfb100d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:02.332240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3689fcc-ce2f-491b-8f15-b3ba1dfb100d admin admin] PATH_INFO: `/v3/projects/0f0f0365fdce40ae865b5babfacea405/users/d82e8329da7b489fa3b084dc1adbf1b0/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:02.332403 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 827/1635] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:02 2022] POST /identity/v3/users => generated 293 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:02.334400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6b101-ae99-4f44-8709-548f4cce43d1 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:02.339003 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b9b6b101-ae99-4f44-8709-548f4cce43d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:02.373402 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3689fcc-ce2f-491b-8f15-b3ba1dfb100d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0f0365fdce40ae865b5babfacea405, user_id=d82e8329da7b489fa3b084dc1adbf1b0, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:02.378271 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3689fcc-ce2f-491b-8f15-b3ba1dfb100d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:02.413444 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 809/1636] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:02 2022] PUT /identity/v3/projects/0f0f0365fdce40ae865b5babfacea405/users/d82e8329da7b489fa3b084dc1adbf1b0/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:02.417913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3689fcc-ce2f-491b-8f15-b3ba1dfb100d admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:02.488321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6b101-ae99-4f44-8709-548f4cce43d1 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:02.577960 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3689fcc-ce2f-491b-8f15-b3ba1dfb100d admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:02.585800 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0aa70-8ba4-412c-ae2e-a0a73c23e0f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:02.587593 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0aa70-8ba4-412c-ae2e-a0a73c23e0f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:02.587593 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0aa70-8ba4-412c-ae2e-a0a73c23e0f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:02.587593 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0aa70-8ba4-412c-ae2e-a0a73c23e0f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:02.589514 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c681e-8aa7-4b4c-8405-d1ef8b7432a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:02.590150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c681e-8aa7-4b4c-8405-d1ef8b7432a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:02.590431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c681e-8aa7-4b4c-8405-d1ef8b7432a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:02.592088 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c681e-8aa7-4b4c-8405-d1ef8b7432a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:02.595528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c681e-8aa7-4b4c-8405-d1ef8b7432a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:02.597166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c681e-8aa7-4b4c-8405-d1ef8b7432a3 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:02.600331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:02.604942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0aa70-8ba4-412c-ae2e-a0a73c23e0f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:02.606390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0aa70-8ba4-412c-ae2e-a0a73c23e0f5 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:02.621418 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 810/1637] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:02.622825 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c681e-8aa7-4b4c-8405-d1ef8b7432a3 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:02.628043 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 828/1638] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:02.628931 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0aa70-8ba4-412c-ae2e-a0a73c23e0f5 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:02.647809 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c681e-8aa7-4b4c-8405-d1ef8b7432a3 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:02.648027 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0aa70-8ba4-412c-ae2e-a0a73c23e0f5 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:02.661969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8e97c-ce21-41ef-b0e0-bb38e8bf8fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:02.663139 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55907034-fa9d-46de-99ed-4ed3da6ec298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:02.663568 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8e97c-ce21-41ef-b0e0-bb38e8bf8fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:02.663694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55907034-fa9d-46de-99ed-4ed3da6ec298 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:02.663858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8e97c-ce21-41ef-b0e0-bb38e8bf8fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:02.663966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55907034-fa9d-46de-99ed-4ed3da6ec298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:02.664178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55907034-fa9d-46de-99ed-4ed3da6ec298 admin admin] PATH_INFO: `/v3/projects/77b7fe6b7e694e2583d754ed11ecb716/users/ea2d7b5b58584822b056ecf5f94df16e/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:02.666255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8e97c-ce21-41ef-b0e0-bb38e8bf8fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:02.667647 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8e97c-ce21-41ef-b0e0-bb38e8bf8fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:02.672790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8e97c-ce21-41ef-b0e0-bb38e8bf8fc1 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:02.698582 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55907034-fa9d-46de-99ed-4ed3da6ec298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b7fe6b7e694e2583d754ed11ecb716, user_id=ea2d7b5b58584822b056ecf5f94df16e, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:02.700055 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 811/1639] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:02.701304 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55907034-fa9d-46de-99ed-4ed3da6ec298 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:02.702982 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8e97c-ce21-41ef-b0e0-bb38e8bf8fc1 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:02.727471 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 829/1640] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:02 2022] PUT /identity/v3/projects/77b7fe6b7e694e2583d754ed11ecb716/users/ea2d7b5b58584822b056ecf5f94df16e/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:02.729233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8e97c-ce21-41ef-b0e0-bb38e8bf8fc1 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:02.731659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55907034-fa9d-46de-99ed-4ed3da6ec298 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:02.823586 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce0abe-3368-4eea-9506-4fce2ce0338d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:02.824514 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce0abe-3368-4eea-9506-4fce2ce0338d admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:02.824864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce0abe-3368-4eea-9506-4fce2ce0338d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:02.825330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce0abe-3368-4eea-9506-4fce2ce0338d admin admin] PATH_INFO: `/v3/projects/0f0f0365fdce40ae865b5babfacea405/users/d82e8329da7b489fa3b084dc1adbf1b0/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:02.827922 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce0abe-3368-4eea-9506-4fce2ce0338d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0f0365fdce40ae865b5babfacea405, user_id=d82e8329da7b489fa3b084dc1adbf1b0, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:02.831225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce0abe-3368-4eea-9506-4fce2ce0338d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:02.840798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55907034-fa9d-46de-99ed-4ed3da6ec298 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:02.846478 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55d3e8-41db-4317-a6f9-03e3dfbb6c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:02.847399 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55d3e8-41db-4317-a6f9-03e3dfbb6c9a admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:02.847863 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55d3e8-41db-4317-a6f9-03e3dfbb6c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:02.848316 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55d3e8-41db-4317-a6f9-03e3dfbb6c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:02.849112 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55d3e8-41db-4317-a6f9-03e3dfbb6c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:02.850923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55d3e8-41db-4317-a6f9-03e3dfbb6c9a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:02.859040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 812/1641] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:02 2022] PUT /identity/v3/projects/0f0f0365fdce40ae865b5babfacea405/users/d82e8329da7b489fa3b084dc1adbf1b0/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:02.862697 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34f18f-06b6-4143-81f6-5ff2bbfb460a None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:02.863000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34f18f-06b6-4143-81f6-5ff2bbfb460a None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:02.863281 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34f18f-06b6-4143-81f6-5ff2bbfb460a None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:02.868031 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 813/1642] 10.222.0.26 () {54 vars in 912 bytes} [Thu Sep 22 14:58:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:02.876330 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 830/1643] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:02.876528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1f380-24ce-4974-a4eb-8570a87a9834 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:02.876528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1f380-24ce-4974-a4eb-8570a87a9834 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:02.876528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1f380-24ce-4974-a4eb-8570a87a9834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:02.879308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55d3e8-41db-4317-a6f9-03e3dfbb6c9a admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:02.897025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55d3e8-41db-4317-a6f9-03e3dfbb6c9a admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:02.920139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-18b1f380-24ce-4974-a4eb-8570a87a9834 None None] MFA Rules not processed for user `0c9be41c39494e54a14398a1f03c4d69`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:02.921559 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-18b1f380-24ce-4974-a4eb-8570a87a9834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:02.968800 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:03.027838 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09520da3-9151-4cb4-aad6-634e96e27b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:03.028525 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09520da3-9151-4cb4-aad6-634e96e27b10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:03.028883 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09520da3-9151-4cb4-aad6-634e96e27b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:03.029166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09520da3-9151-4cb4-aad6-634e96e27b10 admin admin] PATH_INFO: `/v3/projects/77b7fe6b7e694e2583d754ed11ecb716/users/ea2d7b5b58584822b056ecf5f94df16e/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:03.035990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09520da3-9151-4cb4-aad6-634e96e27b10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b7fe6b7e694e2583d754ed11ecb716, user_id=ea2d7b5b58584822b056ecf5f94df16e, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:03.039008 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09520da3-9151-4cb4-aad6-634e96e27b10 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:03.065885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 831/1644] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:02 2022] PUT /identity/v3/projects/77b7fe6b7e694e2583d754ed11ecb716/users/ea2d7b5b58584822b056ecf5f94df16e/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:03.077739 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82d755-def3-435f-8918-38b4372078eb None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:03.078032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82d755-def3-435f-8918-38b4372078eb None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:03.078382 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82d755-def3-435f-8918-38b4372078eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:03.085280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 814/1645] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Sep 22 14:58:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:03.106631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1f380-24ce-4974-a4eb-8570a87a9834 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:03.108220 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-18b1f380-24ce-4974-a4eb-8570a87a9834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:03.129783 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-eb82d755-def3-435f-8918-38b4372078eb None None] MFA Rules not processed for user `ea2d7b5b58584822b056ecf5f94df16e`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:03.130975 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-eb82d755-def3-435f-8918-38b4372078eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:03.214973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1f380-24ce-4974-a4eb-8570a87a9834 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:03.257731 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:03.278662 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 832/1646] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:03.298687 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82d755-def3-435f-8918-38b4372078eb None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:03.300346 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-eb82d755-def3-435f-8918-38b4372078eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:03.321209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea52ec0-ec8a-4552-b4aa-75d0fbfe6b04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:03.324782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea52ec0-ec8a-4552-b4aa-75d0fbfe6b04 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:03.325333 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea52ec0-ec8a-4552-b4aa-75d0fbfe6b04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:03.325333 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea52ec0-ec8a-4552-b4aa-75d0fbfe6b04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:03.381640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82d755-def3-435f-8918-38b4372078eb None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:03.389467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52908ce3-48cf-498b-9cf3-6a15d1916e52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:03.390461 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52908ce3-48cf-498b-9cf3-6a15d1916e52 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:03.391055 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52908ce3-48cf-498b-9cf3-6a15d1916e52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:03.391604 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52908ce3-48cf-498b-9cf3-6a15d1916e52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:03.408756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52908ce3-48cf-498b-9cf3-6a15d1916e52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:03.410147 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52908ce3-48cf-498b-9cf3-6a15d1916e52 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:03.445398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 833/1647] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:03.527257 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea52ec0-ec8a-4552-b4aa-75d0fbfe6b04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:03.528890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea52ec0-ec8a-4552-b4aa-75d0fbfe6b04 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:03.556528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 815/1648] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:03.598845 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52908ce3-48cf-498b-9cf3-6a15d1916e52 service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:03.613861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52908ce3-48cf-498b-9cf3-6a15d1916e52 service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:03.622013 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4c217-99ad-4365-b35b-fda818ea5b5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:03.623034 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4c217-99ad-4365-b35b-fda818ea5b5e service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:03.623580 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4c217-99ad-4365-b35b-fda818ea5b5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:03.624137 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4c217-99ad-4365-b35b-fda818ea5b5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:03.627469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e8c4c217-99ad-4365-b35b-fda818ea5b5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:03.717424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:03.743440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4c217-99ad-4365-b35b-fda818ea5b5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:03.745010 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4c217-99ad-4365-b35b-fda818ea5b5e service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:03.769262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 834/1649] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:05.029473 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7c884-1748-4240-98cd-18a017f86486 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:05.030010 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7c884-1748-4240-98cd-18a017f86486 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:05.030010 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7c884-1748-4240-98cd-18a017f86486 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:05.031805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 816/1650] 10.222.0.26 () {54 vars in 961 bytes} [Thu Sep 22 14:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:05.040032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c1985-cd7c-40ed-829f-8306f5ece8b9 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:05.040630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c1985-cd7c-40ed-829f-8306f5ece8b9 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:05.040630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c1985-cd7c-40ed-829f-8306f5ece8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:05.072297 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-062c1985-cd7c-40ed-829f-8306f5ece8b9 None None] MFA Rules not processed for user `0c9be41c39494e54a14398a1f03c4d69`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:05.074782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-062c1985-cd7c-40ed-829f-8306f5ece8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:05.085640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 835/1651] 10.222.0.26 () {58 vars in 1058 bytes} [Thu Sep 22 14:58:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:05.093042 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d1321-14aa-48bd-9e56-026ff2b0dfef None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:05.093372 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d1321-14aa-48bd-9e56-026ff2b0dfef None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:05.093620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d1321-14aa-48bd-9e56-026ff2b0dfef None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:05.095206 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 817/1652] 10.222.0.26 () {54 vars in 912 bytes} [Thu Sep 22 14:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:05.100883 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343948e6-3ae0-4358-a289-82e4aee31db7 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:05.101280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343948e6-3ae0-4358-a289-82e4aee31db7 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:05.101575 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343948e6-3ae0-4358-a289-82e4aee31db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:05.128854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-343948e6-3ae0-4358-a289-82e4aee31db7 None None] MFA Rules not processed for user `0c9be41c39494e54a14398a1f03c4d69`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:05.130192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-343948e6-3ae0-4358-a289-82e4aee31db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:05.148224 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 836/1653] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Sep 22 14:58:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:05.156531 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d1321-14aa-48bd-9e56-026ff2b0dfef None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:05.171605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d1321-14aa-48bd-9e56-026ff2b0dfef None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:05.197384 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cccb62-489f-426f-97a8-d9e485a479c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:05.198437 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cccb62-489f-426f-97a8-d9e485a479c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:05.198714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cccb62-489f-426f-97a8-d9e485a479c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:05.199016 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cccb62-489f-426f-97a8-d9e485a479c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:05.225134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cccb62-489f-426f-97a8-d9e485a479c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:05.229026 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cccb62-489f-426f-97a8-d9e485a479c6 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:05.264130 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 818/1654] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:05.401717 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343948e6-3ae0-4358-a289-82e4aee31db7 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:05.403204 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-343948e6-3ae0-4358-a289-82e4aee31db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:05.580645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343948e6-3ae0-4358-a289-82e4aee31db7 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:05.593532 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7522f24-fb44-4a3a-883b-b21ac4ad14be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:05.594357 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7522f24-fb44-4a3a-883b-b21ac4ad14be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:05.594639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7522f24-fb44-4a3a-883b-b21ac4ad14be admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:05.596183 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7522f24-fb44-4a3a-883b-b21ac4ad14be admin admin] PATH_INFO: `/v3/projects/5801efe6a126482098daa0760452c327` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:05.598795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7522f24-fb44-4a3a-883b-b21ac4ad14be admin admin] RBAC: Authorizing `identity:delete_project(project_id=5801efe6a126482098daa0760452c327)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:05.599839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: WARNING py.warnings [None req-b7522f24-fb44-4a3a-883b-b21ac4ad14be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 14:58:05.599839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: warnings.warn(msg) Sep 22 14:58:05.599839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: Sep 22 14:58:05.606941 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7522f24-fb44-4a3a-883b-b21ac4ad14be admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:05.783448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b7522f24-fb44-4a3a-883b-b21ac4ad14be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5801efe6a126482098daa0760452c327', 'request_id': 'req-b7522f24-fb44-4a3a-883b-b21ac4ad14be'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:05.800785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b7522f24-fb44-4a3a-883b-b21ac4ad14be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5801efe6a126482098daa0760452c327', 'request_id': 'req-b7522f24-fb44-4a3a-883b-b21ac4ad14be'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:05.831045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b7522f24-fb44-4a3a-883b-b21ac4ad14be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5801efe6a126482098daa0760452c327', 'request_id': 'req-b7522f24-fb44-4a3a-883b-b21ac4ad14be'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:05.862550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b7522f24-fb44-4a3a-883b-b21ac4ad14be admin admin] The token cache is being invalidate because project 5801efe6a126482098daa0760452c327 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:05.862945 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b7522f24-fb44-4a3a-883b-b21ac4ad14be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:05.864409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b7522f24-fb44-4a3a-883b-b21ac4ad14be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:05.871165 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 837/1655] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:05 2022] DELETE /identity/v3/projects/5801efe6a126482098daa0760452c327 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:05.872707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cccb62-489f-426f-97a8-d9e485a479c6 service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:05.892615 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cccb62-489f-426f-97a8-d9e485a479c6 service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:06.075999 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ef0d2-aa4b-4dc5-b14d-06375a91df29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:06.077854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ef0d2-aa4b-4dc5-b14d-06375a91df29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:06.078135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ef0d2-aa4b-4dc5-b14d-06375a91df29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:06.080604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ef0d2-aa4b-4dc5-b14d-06375a91df29 admin admin] PATH_INFO: `/v3/users/768e8cf10fc7410e9b97d7366414b417` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:06.083306 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ef0d2-aa4b-4dc5-b14d-06375a91df29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=768e8cf10fc7410e9b97d7366414b417)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:06.089251 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ef0d2-aa4b-4dc5-b14d-06375a91df29 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:06.196881 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ee3ef0d2-aa4b-4dc5-b14d-06375a91df29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '768e8cf10fc7410e9b97d7366414b417'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:06.217410 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ee3ef0d2-aa4b-4dc5-b14d-06375a91df29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '768e8cf10fc7410e9b97d7366414b417'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:06.220831 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:06.230031 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ee3ef0d2-aa4b-4dc5-b14d-06375a91df29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '768e8cf10fc7410e9b97d7366414b417'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:06.257664 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ee3ef0d2-aa4b-4dc5-b14d-06375a91df29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '768e8cf10fc7410e9b97d7366414b417'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:06.259109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ee3ef0d2-aa4b-4dc5-b14d-06375a91df29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '768e8cf10fc7410e9b97d7366414b417'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:06.290049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 819/1656] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:05 2022] DELETE /identity/v3/users/768e8cf10fc7410e9b97d7366414b417 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:07.096792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908d670-cff9-41dd-bd19-98e576355ea9 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.097304 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908d670-cff9-41dd-bd19-98e576355ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.097304 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908d670-cff9-41dd-bd19-98e576355ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.192602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-9908d670-cff9-41dd-bd19-98e576355ea9 None None] MFA Rules not processed for user `d82e8329da7b489fa3b084dc1adbf1b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:07.193872 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-9908d670-cff9-41dd-bd19-98e576355ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:07.306186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:07.357089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 838/1657] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:07 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:07.581197 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ef0d2-aa4b-4dc5-b14d-06375a91df29 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:07.582754 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-ee3ef0d2-aa4b-4dc5-b14d-06375a91df29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:07.697803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ef0d2-aa4b-4dc5-b14d-06375a91df29 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:07.712675 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed049f6d-38ca-4bee-ab40-b519615a139f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:07.713955 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed049f6d-38ca-4bee-ab40-b519615a139f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.714418 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed049f6d-38ca-4bee-ab40-b519615a139f admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.715099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed049f6d-38ca-4bee-ab40-b519615a139f admin admin] PATH_INFO: `/v3/projects/3ee9b835747f43bb9c5b8392301bb4ce` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.717271 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed049f6d-38ca-4bee-ab40-b519615a139f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ee9b835747f43bb9c5b8392301bb4ce)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:07.719719 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed049f6d-38ca-4bee-ab40-b519615a139f admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:07.871167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ed049f6d-38ca-4bee-ab40-b519615a139f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ee9b835747f43bb9c5b8392301bb4ce', 'request_id': 'req-ed049f6d-38ca-4bee-ab40-b519615a139f'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:07.889541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ed049f6d-38ca-4bee-ab40-b519615a139f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ee9b835747f43bb9c5b8392301bb4ce', 'request_id': 'req-ed049f6d-38ca-4bee-ab40-b519615a139f'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:07.905057 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ed049f6d-38ca-4bee-ab40-b519615a139f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ee9b835747f43bb9c5b8392301bb4ce', 'request_id': 'req-ed049f6d-38ca-4bee-ab40-b519615a139f'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:07.927115 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ed049f6d-38ca-4bee-ab40-b519615a139f admin admin] The token cache is being invalidate because project 3ee9b835747f43bb9c5b8392301bb4ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:07.927445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ed049f6d-38ca-4bee-ab40-b519615a139f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:07.931037 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ed049f6d-38ca-4bee-ab40-b519615a139f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:07.934078 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 820/1658] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:07 2022] DELETE /identity/v3/projects/3ee9b835747f43bb9c5b8392301bb4ce => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:07.939840 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf5ae0-3f59-4961-baf0-2f72b849eb79 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:07.940131 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf5ae0-3f59-4961-baf0-2f72b849eb79 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:07.940424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf5ae0-3f59-4961-baf0-2f72b849eb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:07.984405 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-eebf5ae0-3f59-4961-baf0-2f72b849eb79 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:07.986554 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-eebf5ae0-3f59-4961-baf0-2f72b849eb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:08.133822 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 839/1659] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:08.143447 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71745e-2f00-4fe7-b2b0-d4cf0ef750c1 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.144548 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71745e-2f00-4fe7-b2b0-d4cf0ef750c1 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.145103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71745e-2f00-4fe7-b2b0-d4cf0ef750c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.196085 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-0e71745e-2f00-4fe7-b2b0-d4cf0ef750c1 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:08.197900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0e71745e-2f00-4fe7-b2b0-d4cf0ef750c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:08.216520 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 821/1660] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:08.219232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf5ae0-3f59-4961-baf0-2f72b849eb79 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:08.239312 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf5ae0-3f59-4961-baf0-2f72b849eb79 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.246466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3808df1-3b3b-459b-8a9e-912bf8411d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.247585 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3808df1-3b3b-459b-8a9e-912bf8411d94 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.248142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3808df1-3b3b-459b-8a9e-912bf8411d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.248708 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3808df1-3b3b-459b-8a9e-912bf8411d94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.249734 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3808df1-3b3b-459b-8a9e-912bf8411d94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:08.250174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3808df1-3b3b-459b-8a9e-912bf8411d94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.254069 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3808df1-3b3b-459b-8a9e-912bf8411d94 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.281217 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 840/1661] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:58:08.285408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71745e-2f00-4fe7-b2b0-d4cf0ef750c1 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:08.305777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71745e-2f00-4fe7-b2b0-d4cf0ef750c1 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.324979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90e473-afc0-4017-9873-eb5a22e7f26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.325884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90e473-afc0-4017-9873-eb5a22e7f26e admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.326297 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90e473-afc0-4017-9873-eb5a22e7f26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.326644 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90e473-afc0-4017-9873-eb5a22e7f26e admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.327421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90e473-afc0-4017-9873-eb5a22e7f26e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.335344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90e473-afc0-4017-9873-eb5a22e7f26e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.380977 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 822/1662] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:08 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:08.382188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3808df1-3b3b-459b-8a9e-912bf8411d94 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:08.417484 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3808df1-3b3b-459b-8a9e-912bf8411d94 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.603479 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224c301-820b-48b4-9124-eeb7b5d0b8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.604860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224c301-820b-48b4-9124-eeb7b5d0b8c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.607941 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224c301-820b-48b4-9124-eeb7b5d0b8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.608490 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224c301-820b-48b4-9124-eeb7b5d0b8c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.609676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224c301-820b-48b4-9124-eeb7b5d0b8c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.616807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224c301-820b-48b4-9124-eeb7b5d0b8c9 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.691891 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 841/1663] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:08 2022] POST /identity/v3/users => generated 293 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:08.694443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90e473-afc0-4017-9873-eb5a22e7f26e admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:08.716768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90e473-afc0-4017-9873-eb5a22e7f26e admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.723686 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fae2a-62a1-4409-9aa7-3da791977030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.724756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fae2a-62a1-4409-9aa7-3da791977030 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.725213 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fae2a-62a1-4409-9aa7-3da791977030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.725749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fae2a-62a1-4409-9aa7-3da791977030 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.726683 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fae2a-62a1-4409-9aa7-3da791977030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.728848 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fae2a-62a1-4409-9aa7-3da791977030 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.747452 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224c301-820b-48b4-9124-eeb7b5d0b8c9 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:08.755684 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 823/1664] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:08.763393 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224c301-820b-48b4-9124-eeb7b5d0b8c9 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.771661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215c4c0-7686-4014-a22e-297803acb453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.773288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215c4c0-7686-4014-a22e-297803acb453 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.774374 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215c4c0-7686-4014-a22e-297803acb453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.775506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215c4c0-7686-4014-a22e-297803acb453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.776475 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d215c4c0-7686-4014-a22e-297803acb453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.778568 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d215c4c0-7686-4014-a22e-297803acb453 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.798077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fae2a-62a1-4409-9aa7-3da791977030 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:08.802189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 842/1665] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:08.815482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fae2a-62a1-4409-9aa7-3da791977030 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:08.821828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a98f9c-6d89-44f1-87e9-6343c169c782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:08.822527 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a98f9c-6d89-44f1-87e9-6343c169c782 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.822798 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a98f9c-6d89-44f1-87e9-6343c169c782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.823073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a98f9c-6d89-44f1-87e9-6343c169c782 admin admin] PATH_INFO: `/v3/projects/76e5f873e1cc49bcb84c470f4e71de73/users/8baaffad42bf4090a29b280f40377dfd/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.848326 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a98f9c-6d89-44f1-87e9-6343c169c782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76e5f873e1cc49bcb84c470f4e71de73, user_id=8baaffad42bf4090a29b280f40377dfd, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:08.851670 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a98f9c-6d89-44f1-87e9-6343c169c782 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:08.881919 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:08.898267 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 824/1666] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:08 2022] PUT /identity/v3/projects/76e5f873e1cc49bcb84c470f4e71de73/users/8baaffad42bf4090a29b280f40377dfd/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:08.910116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be78b4-345d-4f22-8d2f-a62e5b365ee7 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:08.910399 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be78b4-345d-4f22-8d2f-a62e5b365ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:08.910676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be78b4-345d-4f22-8d2f-a62e5b365ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:08.973637 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-d9be78b4-345d-4f22-8d2f-a62e5b365ee7 None None] MFA Rules not processed for user `8baaffad42bf4090a29b280f40377dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:08.974379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d9be78b4-345d-4f22-8d2f-a62e5b365ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:09.069781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:09.163424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 843/1667] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:09.168179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a98f9c-6d89-44f1-87e9-6343c169c782 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:09.366876 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a98f9c-6d89-44f1-87e9-6343c169c782 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:09.385237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5460042-d1af-4770-b8e5-b5b28b3174f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:09.386113 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5460042-d1af-4770-b8e5-b5b28b3174f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.387843 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5460042-d1af-4770-b8e5-b5b28b3174f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.388963 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5460042-d1af-4770-b8e5-b5b28b3174f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.392410 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5460042-d1af-4770-b8e5-b5b28b3174f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:09.395205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5460042-d1af-4770-b8e5-b5b28b3174f0 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:09.464972 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 825/1668] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:09 2022] POST /identity/v3/projects => generated 387 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:09.469462 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be78b4-345d-4f22-8d2f-a62e5b365ee7 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:09.484611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be78b4-345d-4f22-8d2f-a62e5b365ee7 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:09.608682 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff08b9-c794-4b6d-b83a-2ae068b793d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:09.608984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff08b9-c794-4b6d-b83a-2ae068b793d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.608984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff08b9-c794-4b6d-b83a-2ae068b793d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.609269 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff08b9-c794-4b6d-b83a-2ae068b793d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.609915 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ff08b9-c794-4b6d-b83a-2ae068b793d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:09.612774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ff08b9-c794-4b6d-b83a-2ae068b793d5 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:09.627491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7397c-7980-4a9c-9dde-607b637f2d29 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.627783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7397c-7980-4a9c-9dde-607b637f2d29 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.628152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7397c-7980-4a9c-9dde-607b637f2d29 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.630702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 826/1669] 10.222.0.26 () {54 vars in 921 bytes} [Thu Sep 22 14:58:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:09.654038 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a728b-6ef0-4e24-b8f8-5e89c7ad8c8b None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.654334 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a728b-6ef0-4e24-b8f8-5e89c7ad8c8b None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.654651 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a728b-6ef0-4e24-b8f8-5e89c7ad8c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.699183 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 844/1670] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:09 2022] POST /identity/v3/users => generated 300 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:09.703145 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff08b9-c794-4b6d-b83a-2ae068b793d5 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:09.713760 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-437a728b-6ef0-4e24-b8f8-5e89c7ad8c8b None None] MFA Rules not processed for user `47b6b34b4ee5492fb1f0699716e5118b`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:09.719240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-437a728b-6ef0-4e24-b8f8-5e89c7ad8c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:09.734987 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff08b9-c794-4b6d-b83a-2ae068b793d5 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:09.755415 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c733029-1b32-4462-8702-9448b94f592a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:09.759511 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c733029-1b32-4462-8702-9448b94f592a admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.759511 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c733029-1b32-4462-8702-9448b94f592a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.761206 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c733029-1b32-4462-8702-9448b94f592a admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.763531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c733029-1b32-4462-8702-9448b94f592a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:09.765791 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c733029-1b32-4462-8702-9448b94f592a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:09.803395 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 845/1671] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:09.808078 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c733029-1b32-4462-8702-9448b94f592a admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:09.835116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c733029-1b32-4462-8702-9448b94f592a admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:09.846934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34396919-147a-4346-8065-dac5ffe35c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:09.850471 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34396919-147a-4346-8065-dac5ffe35c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:09.850869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34396919-147a-4346-8065-dac5ffe35c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:09.851156 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34396919-147a-4346-8065-dac5ffe35c97 admin admin] PATH_INFO: `/v3/projects/9453e93adacd4319b43bdf540f9ec572/users/c247c1c112034edb8a12b2489ebbabb5/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:09.898710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34396919-147a-4346-8065-dac5ffe35c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9453e93adacd4319b43bdf540f9ec572, user_id=c247c1c112034edb8a12b2489ebbabb5, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:09.907777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34396919-147a-4346-8065-dac5ffe35c97 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:09.927876 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 827/1672] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Sep 22 14:58:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:09.939691 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 846/1673] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:09 2022] PUT /identity/v3/projects/9453e93adacd4319b43bdf540f9ec572/users/c247c1c112034edb8a12b2489ebbabb5/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:09.943316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a728b-6ef0-4e24-b8f8-5e89c7ad8c8b None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:09.960438 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a728b-6ef0-4e24-b8f8-5e89c7ad8c8b None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:10.077909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34396919-147a-4346-8065-dac5ffe35c97 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:10.080701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-34396919-147a-4346-8065-dac5ffe35c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:10.110377 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb67ec-19e5-4a98-8f9d-e3dd09e3ce8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:10.111000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb67ec-19e5-4a98-8f9d-e3dd09e3ce8f admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:10.111276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb67ec-19e5-4a98-8f9d-e3dd09e3ce8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:10.111544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb67ec-19e5-4a98-8f9d-e3dd09e3ce8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:10.112220 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb67ec-19e5-4a98-8f9d-e3dd09e3ce8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:10.113976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb67ec-19e5-4a98-8f9d-e3dd09e3ce8f admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:10.155187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 828/1674] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:10.156987 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb67ec-19e5-4a98-8f9d-e3dd09e3ce8f admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:10.169048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb67ec-19e5-4a98-8f9d-e3dd09e3ce8f admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:10.186542 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ba15f-e9dd-47e0-b6ac-e7b6357a468c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:10.187374 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ba15f-e9dd-47e0-b6ac-e7b6357a468c admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:10.187538 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ba15f-e9dd-47e0-b6ac-e7b6357a468c admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:10.187821 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ba15f-e9dd-47e0-b6ac-e7b6357a468c admin admin] PATH_INFO: `/v3/projects/9453e93adacd4319b43bdf540f9ec572/users/c247c1c112034edb8a12b2489ebbabb5/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:10.196937 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ba15f-e9dd-47e0-b6ac-e7b6357a468c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9453e93adacd4319b43bdf540f9ec572, user_id=c247c1c112034edb8a12b2489ebbabb5, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:10.199707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ba15f-e9dd-47e0-b6ac-e7b6357a468c admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:10.245637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 829/1675] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:10 2022] PUT /identity/v3/projects/9453e93adacd4319b43bdf540f9ec572/users/c247c1c112034edb8a12b2489ebbabb5/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:10.250614 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed608b-1b31-483a-b6f2-3bb5c7a92260 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:10.250955 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed608b-1b31-483a-b6f2-3bb5c7a92260 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:10.251452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed608b-1b31-483a-b6f2-3bb5c7a92260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:10.270551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34396919-147a-4346-8065-dac5ffe35c97 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:10.275517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:10.310343 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-d6ed608b-1b31-483a-b6f2-3bb5c7a92260 None None] MFA Rules not processed for user `c247c1c112034edb8a12b2489ebbabb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:10.313325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d6ed608b-1b31-483a-b6f2-3bb5c7a92260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:10.355657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:10.384808 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18975377-3670-488b-b6dc-49324c2d2a6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b6b34b4ee5492fb1f0699716e5118b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:10.385448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18975377-3670-488b-b6dc-49324c2d2a6f service placement] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:10.385797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18975377-3670-488b-b6dc-49324c2d2a6f service placement] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:10.386076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18975377-3670-488b-b6dc-49324c2d2a6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:10.398451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18975377-3670-488b-b6dc-49324c2d2a6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:10.400619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18975377-3670-488b-b6dc-49324c2d2a6f service placement] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:10.455658 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 847/1676] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:10.533553 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 830/1677] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:10.545608 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18975377-3670-488b-b6dc-49324c2d2a6f service placement] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:10.548741 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-18975377-3670-488b-b6dc-49324c2d2a6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:10.734452 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18975377-3670-488b-b6dc-49324c2d2a6f service placement] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:10.744736 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e5cec-0038-4cb7-9b65-e062372e720a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:10.745423 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e5cec-0038-4cb7-9b65-e062372e720a service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:10.745650 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e5cec-0038-4cb7-9b65-e062372e720a service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:10.745921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e5cec-0038-4cb7-9b65-e062372e720a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:10.767154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e5cec-0038-4cb7-9b65-e062372e720a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:10.768948 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e5cec-0038-4cb7-9b65-e062372e720a service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:10.797987 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 848/1678] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:10 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:10.941478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed608b-1b31-483a-b6f2-3bb5c7a92260 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:11.063482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed608b-1b31-483a-b6f2-3bb5c7a92260 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:11.069194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6ce99-687b-433b-a3b3-ecfeccc4e70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:11.076087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6ce99-687b-433b-a3b3-ecfeccc4e70a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:11.076553 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6ce99-687b-433b-a3b3-ecfeccc4e70a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:11.076900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6ce99-687b-433b-a3b3-ecfeccc4e70a admin admin] PATH_INFO: `/v3/users/8baaffad42bf4090a29b280f40377dfd` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:11.079573 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6ce99-687b-433b-a3b3-ecfeccc4e70a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8baaffad42bf4090a29b280f40377dfd)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:11.081773 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6ce99-687b-433b-a3b3-ecfeccc4e70a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:11.176478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0fc6ce99-687b-433b-a3b3-ecfeccc4e70a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8baaffad42bf4090a29b280f40377dfd'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:11.189483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0fc6ce99-687b-433b-a3b3-ecfeccc4e70a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8baaffad42bf4090a29b280f40377dfd'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:11.244115 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0fc6ce99-687b-433b-a3b3-ecfeccc4e70a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8baaffad42bf4090a29b280f40377dfd'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:11.271477 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0fc6ce99-687b-433b-a3b3-ecfeccc4e70a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8baaffad42bf4090a29b280f40377dfd'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:11.272381 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0fc6ce99-687b-433b-a3b3-ecfeccc4e70a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8baaffad42bf4090a29b280f40377dfd'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:11.283833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 831/1679] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:10 2022] DELETE /identity/v3/users/8baaffad42bf4090a29b280f40377dfd => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:11.287193 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1bd95-5e9b-4554-b453-3c0d785c5150 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:11.287659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1bd95-5e9b-4554-b453-3c0d785c5150 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:11.287730 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1bd95-5e9b-4554-b453-3c0d785c5150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:11.320615 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-14a1bd95-5e9b-4554-b453-3c0d785c5150 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:11.321869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-14a1bd95-5e9b-4554-b453-3c0d785c5150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:11.447425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 849/1680] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:11.472169 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6ce99-687b-433b-a3b3-ecfeccc4e70a admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:11.473833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0fc6ce99-687b-433b-a3b3-ecfeccc4e70a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:11.512857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:11.622476 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6ce99-687b-433b-a3b3-ecfeccc4e70a admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:11.632950 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c91727-9486-4e4e-87f3-2554379afc57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:11.633974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c91727-9486-4e4e-87f3-2554379afc57 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:11.634356 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c91727-9486-4e4e-87f3-2554379afc57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:11.634756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c91727-9486-4e4e-87f3-2554379afc57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:11.660483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c91727-9486-4e4e-87f3-2554379afc57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:11.662122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c91727-9486-4e4e-87f3-2554379afc57 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:11.699398 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 832/1681] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:13.009189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e5a5c-83ef-4225-ae7e-efb4348e226b None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:13.010152 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e5a5c-83ef-4225-ae7e-efb4348e226b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:13.010568 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e5a5c-83ef-4225-ae7e-efb4348e226b None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:13.012144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 850/1682] 10.222.0.26 () {54 vars in 921 bytes} [Thu Sep 22 14:58:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:13.172769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b882f-a5c6-4c5a-ad92-b9d3ec7654d8 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:13.173277 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b882f-a5c6-4c5a-ad92-b9d3ec7654d8 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:13.174422 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b882f-a5c6-4c5a-ad92-b9d3ec7654d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:13.209214 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-e47b882f-a5c6-4c5a-ad92-b9d3ec7654d8 None None] MFA Rules not processed for user `47b6b34b4ee5492fb1f0699716e5118b`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:13.215189 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e47b882f-a5c6-4c5a-ad92-b9d3ec7654d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:13.291133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e5a5c-83ef-4225-ae7e-efb4348e226b None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:13.294680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d22e5a5c-83ef-4225-ae7e-efb4348e226b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:13.341744 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e5a5c-83ef-4225-ae7e-efb4348e226b None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:13.353194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af977e-6630-4b57-b02a-6858fe94cb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:13.353839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af977e-6630-4b57-b02a-6858fe94cb8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:13.354113 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af977e-6630-4b57-b02a-6858fe94cb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:13.355592 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af977e-6630-4b57-b02a-6858fe94cb8f admin admin] PATH_INFO: `/v3/users/0956e9529ad544f6a2ae8cdcbd56ab3d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:13.358011 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af977e-6630-4b57-b02a-6858fe94cb8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0956e9529ad544f6a2ae8cdcbd56ab3d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:13.363624 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af977e-6630-4b57-b02a-6858fe94cb8f admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:13.434156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 833/1683] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Sep 22 14:58:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:13.435575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b882f-a5c6-4c5a-ad92-b9d3ec7654d8 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:13.436982 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e47b882f-a5c6-4c5a-ad92-b9d3ec7654d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:13.490722 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b4af977e-6630-4b57-b02a-6858fe94cb8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0956e9529ad544f6a2ae8cdcbd56ab3d'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:13.493461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b882f-a5c6-4c5a-ad92-b9d3ec7654d8 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:13.502574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794acf3-7b4d-4808-90db-a6e07a5d15b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:13.503637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794acf3-7b4d-4808-90db-a6e07a5d15b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:13.503964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794acf3-7b4d-4808-90db-a6e07a5d15b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:13.504285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794acf3-7b4d-4808-90db-a6e07a5d15b9 admin admin] PATH_INFO: `/v3/projects/76e5f873e1cc49bcb84c470f4e71de73` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:13.506478 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b4af977e-6630-4b57-b02a-6858fe94cb8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0956e9529ad544f6a2ae8cdcbd56ab3d'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:13.508430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b4af977e-6630-4b57-b02a-6858fe94cb8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0956e9529ad544f6a2ae8cdcbd56ab3d'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:13.513587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794acf3-7b4d-4808-90db-a6e07a5d15b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76e5f873e1cc49bcb84c470f4e71de73)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:13.519278 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794acf3-7b4d-4808-90db-a6e07a5d15b9 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:13.540459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b4af977e-6630-4b57-b02a-6858fe94cb8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0956e9529ad544f6a2ae8cdcbd56ab3d'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:13.541934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b4af977e-6630-4b57-b02a-6858fe94cb8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0956e9529ad544f6a2ae8cdcbd56ab3d'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:13.544257 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:13.577468 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 851/1684] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:13 2022] DELETE /identity/v3/users/0956e9529ad544f6a2ae8cdcbd56ab3d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:13.579843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af977e-6630-4b57-b02a-6858fe94cb8f admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:13.584093 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b4af977e-6630-4b57-b02a-6858fe94cb8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:13.635275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3794acf3-7b4d-4808-90db-a6e07a5d15b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76e5f873e1cc49bcb84c470f4e71de73', 'request_id': 'req-3794acf3-7b4d-4808-90db-a6e07a5d15b9'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:13.650933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3794acf3-7b4d-4808-90db-a6e07a5d15b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76e5f873e1cc49bcb84c470f4e71de73', 'request_id': 'req-3794acf3-7b4d-4808-90db-a6e07a5d15b9'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:13.675069 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3794acf3-7b4d-4808-90db-a6e07a5d15b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76e5f873e1cc49bcb84c470f4e71de73', 'request_id': 'req-3794acf3-7b4d-4808-90db-a6e07a5d15b9'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:13.700865 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3794acf3-7b4d-4808-90db-a6e07a5d15b9 admin admin] The token cache is being invalidate because project 76e5f873e1cc49bcb84c470f4e71de73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:13.701156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3794acf3-7b4d-4808-90db-a6e07a5d15b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:13.709805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3794acf3-7b4d-4808-90db-a6e07a5d15b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:13.711737 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 834/1685] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:13 2022] DELETE /identity/v3/projects/76e5f873e1cc49bcb84c470f4e71de73 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:13.715875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794acf3-7b4d-4808-90db-a6e07a5d15b9 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:13.717792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3794acf3-7b4d-4808-90db-a6e07a5d15b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:13.797280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:13.802169 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af977e-6630-4b57-b02a-6858fe94cb8f admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:13.866289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794acf3-7b4d-4808-90db-a6e07a5d15b9 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:13.883262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e210c-dc96-4878-8377-e6bba96f15ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:13.883992 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e210c-dc96-4878-8377-e6bba96f15ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:13.884200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e210c-dc96-4878-8377-e6bba96f15ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:13.884469 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e210c-dc96-4878-8377-e6bba96f15ba admin admin] PATH_INFO: `/v3/users/c247c1c112034edb8a12b2489ebbabb5` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:13.887288 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e210c-dc96-4878-8377-e6bba96f15ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=c247c1c112034edb8a12b2489ebbabb5)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:13.891384 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e210c-dc96-4878-8377-e6bba96f15ba admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:13.946354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67931281-4270-4bfe-832d-0eca9867633a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b6b34b4ee5492fb1f0699716e5118b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:13.949013 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67931281-4270-4bfe-832d-0eca9867633a service placement] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:13.949996 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67931281-4270-4bfe-832d-0eca9867633a service placement] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:13.950931 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67931281-4270-4bfe-832d-0eca9867633a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:13.955224 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-67931281-4270-4bfe-832d-0eca9867633a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:13.990548 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67931281-4270-4bfe-832d-0eca9867633a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:13.992903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67931281-4270-4bfe-832d-0eca9867633a service placement] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:13.996147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a85e210c-dc96-4878-8377-e6bba96f15ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c247c1c112034edb8a12b2489ebbabb5'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:14.014251 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a85e210c-dc96-4878-8377-e6bba96f15ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c247c1c112034edb8a12b2489ebbabb5'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:14.026527 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 852/1686] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:14.029122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a85e210c-dc96-4878-8377-e6bba96f15ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c247c1c112034edb8a12b2489ebbabb5'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:14.074214 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a85e210c-dc96-4878-8377-e6bba96f15ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c247c1c112034edb8a12b2489ebbabb5'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:14.075310 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a85e210c-dc96-4878-8377-e6bba96f15ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c247c1c112034edb8a12b2489ebbabb5'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:14.087461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 835/1687] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:13 2022] DELETE /identity/v3/users/c247c1c112034edb8a12b2489ebbabb5 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:15.060054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67931281-4270-4bfe-832d-0eca9867633a service placement] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:15.062393 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-67931281-4270-4bfe-832d-0eca9867633a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:15.107907 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67931281-4270-4bfe-832d-0eca9867633a service placement] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:15.284669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4aad63-fb59-45ba-a6da-f7695a40fd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:15.285334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4aad63-fb59-45ba-a6da-f7695a40fd18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:15.285609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4aad63-fb59-45ba-a6da-f7695a40fd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:15.285880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4aad63-fb59-45ba-a6da-f7695a40fd18 admin admin] PATH_INFO: `/v3/projects/5ff35d1955d84cb3abef1c748996b844` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:15.287446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4aad63-fb59-45ba-a6da-f7695a40fd18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ff35d1955d84cb3abef1c748996b844)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:15.289593 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4aad63-fb59-45ba-a6da-f7695a40fd18 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:15.369103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:15.419199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9f4aad63-fb59-45ba-a6da-f7695a40fd18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ff35d1955d84cb3abef1c748996b844', 'request_id': 'req-9f4aad63-fb59-45ba-a6da-f7695a40fd18'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:15.444291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9f4aad63-fb59-45ba-a6da-f7695a40fd18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ff35d1955d84cb3abef1c748996b844', 'request_id': 'req-9f4aad63-fb59-45ba-a6da-f7695a40fd18'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:15.462446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9f4aad63-fb59-45ba-a6da-f7695a40fd18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ff35d1955d84cb3abef1c748996b844', 'request_id': 'req-9f4aad63-fb59-45ba-a6da-f7695a40fd18'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:15.503125 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9f4aad63-fb59-45ba-a6da-f7695a40fd18 admin admin] The token cache is being invalidate because project 5ff35d1955d84cb3abef1c748996b844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:15.503630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9f4aad63-fb59-45ba-a6da-f7695a40fd18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:15.505119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9f4aad63-fb59-45ba-a6da-f7695a40fd18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:15.515830 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e210c-dc96-4878-8377-e6bba96f15ba admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:15.517130 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 853/1688] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:15 2022] DELETE /identity/v3/projects/5ff35d1955d84cb3abef1c748996b844 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:15.517747 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a85e210c-dc96-4878-8377-e6bba96f15ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:15.572689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e210c-dc96-4878-8377-e6bba96f15ba admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:15.785005 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa9cce7-7b3d-4c45-ac61-f7fb5593cae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:15.785686 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9cce7-7b3d-4c45-ac61-f7fb5593cae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:15.785966 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9cce7-7b3d-4c45-ac61-f7fb5593cae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:15.786239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa9cce7-7b3d-4c45-ac61-f7fb5593cae3 admin admin] PATH_INFO: `/v3/users/d82e8329da7b489fa3b084dc1adbf1b0` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:15.792311 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9cce7-7b3d-4c45-ac61-f7fb5593cae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d82e8329da7b489fa3b084dc1adbf1b0)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:15.795038 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa9cce7-7b3d-4c45-ac61-f7fb5593cae3 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:15.833326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4aad63-fb59-45ba-a6da-f7695a40fd18 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:15.835153 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-9f4aad63-fb59-45ba-a6da-f7695a40fd18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:15.859681 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4aad63-fb59-45ba-a6da-f7695a40fd18 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:15.871493 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5aa715-d19d-4d03-8780-01237782681a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:15.872634 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5aa715-d19d-4d03-8780-01237782681a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:15.873228 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5aa715-d19d-4d03-8780-01237782681a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:15.873704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5aa715-d19d-4d03-8780-01237782681a admin admin] PATH_INFO: `/v3/users/2c38e2a135634a0fb0fb735c61cabf59` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:15.875262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5aa715-d19d-4d03-8780-01237782681a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c38e2a135634a0fb0fb735c61cabf59)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:15.878419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5aa715-d19d-4d03-8780-01237782681a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:15.932118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ffa9cce7-7b3d-4c45-ac61-f7fb5593cae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd82e8329da7b489fa3b084dc1adbf1b0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:15.937626 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:15.949475 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ffa9cce7-7b3d-4c45-ac61-f7fb5593cae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd82e8329da7b489fa3b084dc1adbf1b0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:15.973659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ffa9cce7-7b3d-4c45-ac61-f7fb5593cae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd82e8329da7b489fa3b084dc1adbf1b0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:16.005306 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5d5aa715-d19d-4d03-8780-01237782681a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c38e2a135634a0fb0fb735c61cabf59'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:16.021173 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ffa9cce7-7b3d-4c45-ac61-f7fb5593cae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd82e8329da7b489fa3b084dc1adbf1b0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:16.022222 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ffa9cce7-7b3d-4c45-ac61-f7fb5593cae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd82e8329da7b489fa3b084dc1adbf1b0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:16.056549 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5d5aa715-d19d-4d03-8780-01237782681a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c38e2a135634a0fb0fb735c61cabf59'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:16.057326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5d5aa715-d19d-4d03-8780-01237782681a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c38e2a135634a0fb0fb735c61cabf59'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:16.063180 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 836/1689] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:15 2022] DELETE /identity/v3/users/d82e8329da7b489fa3b084dc1adbf1b0 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:16.083612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5d5aa715-d19d-4d03-8780-01237782681a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c38e2a135634a0fb0fb735c61cabf59'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:16.084999 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5d5aa715-d19d-4d03-8780-01237782681a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c38e2a135634a0fb0fb735c61cabf59'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:16.125701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 854/1690] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:15 2022] DELETE /identity/v3/users/2c38e2a135634a0fb0fb735c61cabf59 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:16.130625 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208b0d5-734e-4530-bd8d-5e79db672ccd None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:16.131338 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208b0d5-734e-4530-bd8d-5e79db672ccd None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:16.131620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208b0d5-734e-4530-bd8d-5e79db672ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:16.186560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-e208b0d5-734e-4530-bd8d-5e79db672ccd None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:16.188675 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e208b0d5-734e-4530-bd8d-5e79db672ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:16.391648 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 837/1691] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:16.399062 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5aa715-d19d-4d03-8780-01237782681a admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:16.404893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-5d5aa715-d19d-4d03-8780-01237782681a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:16.540115 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5aa715-d19d-4d03-8780-01237782681a admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:16.543521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad2141-6807-4a95-b2cb-73e0c68c6493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:16.544158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad2141-6807-4a95-b2cb-73e0c68c6493 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:16.544433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad2141-6807-4a95-b2cb-73e0c68c6493 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:16.544701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad2141-6807-4a95-b2cb-73e0c68c6493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:16.569563 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad2141-6807-4a95-b2cb-73e0c68c6493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:16.572083 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ad2141-6807-4a95-b2cb-73e0c68c6493 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:16.590586 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 855/1692] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:16.806948 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c819c-06bf-47e9-b0e4-359ac414c974 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:16.807527 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c819c-06bf-47e9-b0e4-359ac414c974 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:16.807527 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c819c-06bf-47e9-b0e4-359ac414c974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:16.877773 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-071c819c-06bf-47e9-b0e4-359ac414c974 None None] MFA Rules not processed for user `038365d6ba7041f4af386767d98ed709`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:16.878328 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-071c819c-06bf-47e9-b0e4-359ac414c974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:17.032324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af663a8-e3e7-44cf-9f6e-ecf34fde02dd None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:17.032778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af663a8-e3e7-44cf-9f6e-ecf34fde02dd None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:17.032878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af663a8-e3e7-44cf-9f6e-ecf34fde02dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:17.037608 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 838/1693] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:17.073361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-2af663a8-e3e7-44cf-9f6e-ecf34fde02dd None None] MFA Rules not processed for user `038365d6ba7041f4af386767d98ed709`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:17.075952 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-2af663a8-e3e7-44cf-9f6e-ecf34fde02dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:17.094346 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 856/1694] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:17.105444 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c819c-06bf-47e9-b0e4-359ac414c974 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:17.107030 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-071c819c-06bf-47e9-b0e4-359ac414c974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:17.127893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af663a8-e3e7-44cf-9f6e-ecf34fde02dd None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:17.129120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-2af663a8-e3e7-44cf-9f6e-ecf34fde02dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:17.162834 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af663a8-e3e7-44cf-9f6e-ecf34fde02dd None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:17.184415 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7168f176-d0b1-44cc-852b-75139e1bf143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:17.193847 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168f176-d0b1-44cc-852b-75139e1bf143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:17.193847 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168f176-d0b1-44cc-852b-75139e1bf143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:17.193847 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7168f176-d0b1-44cc-852b-75139e1bf143 admin admin] PATH_INFO: `/v3/projects/9453e93adacd4319b43bdf540f9ec572` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:17.193847 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7168f176-d0b1-44cc-852b-75139e1bf143 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9453e93adacd4319b43bdf540f9ec572)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:17.194720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7168f176-d0b1-44cc-852b-75139e1bf143 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:17.251947 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:17.313353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:17.388437 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-7168f176-d0b1-44cc-852b-75139e1bf143 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9453e93adacd4319b43bdf540f9ec572', 'request_id': 'req-7168f176-d0b1-44cc-852b-75139e1bf143'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:17.391184 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c819c-06bf-47e9-b0e4-359ac414c974 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:17.406440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-7168f176-d0b1-44cc-852b-75139e1bf143 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9453e93adacd4319b43bdf540f9ec572', 'request_id': 'req-7168f176-d0b1-44cc-852b-75139e1bf143'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:17.421718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-7168f176-d0b1-44cc-852b-75139e1bf143 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9453e93adacd4319b43bdf540f9ec572', 'request_id': 'req-7168f176-d0b1-44cc-852b-75139e1bf143'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:17.452834 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-7168f176-d0b1-44cc-852b-75139e1bf143 admin admin] The token cache is being invalidate because project 9453e93adacd4319b43bdf540f9ec572 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:17.457845 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-7168f176-d0b1-44cc-852b-75139e1bf143 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:17.458044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-7168f176-d0b1-44cc-852b-75139e1bf143 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:17.460691 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 857/1695] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:17 2022] DELETE /identity/v3/projects/9453e93adacd4319b43bdf540f9ec572 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:17.467045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1268625-14fb-481c-9b3c-e05486612dfd None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:17.467607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1268625-14fb-481c-9b3c-e05486612dfd None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:17.468506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1268625-14fb-481c-9b3c-e05486612dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:17.574589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5777c32-36c2-4125-96a9-8925a1c730a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:17.575282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5777c32-36c2-4125-96a9-8925a1c730a1 service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:17.575557 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5777c32-36c2-4125-96a9-8925a1c730a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:17.575828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5777c32-36c2-4125-96a9-8925a1c730a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:17.581865 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f5777c32-36c2-4125-96a9-8925a1c730a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:17.585217 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-f1268625-14fb-481c-9b3c-e05486612dfd None None] MFA Rules not processed for user `f298f0c6a9e44efba2a1e2fc648a0f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:17.587087 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-f1268625-14fb-481c-9b3c-e05486612dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:17.783848 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5777c32-36c2-4125-96a9-8925a1c730a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:17.785667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5777c32-36c2-4125-96a9-8925a1c730a1 service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:17.822301 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 839/1696] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 717 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:17.825471 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 858/1697] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:17.826620 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1268625-14fb-481c-9b3c-e05486612dfd None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:17.836405 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-f1268625-14fb-481c-9b3c-e05486612dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:17.846945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1fcf1-0115-4670-9c9e-332c04051aa6 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:17.848182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1fcf1-0115-4670-9c9e-332c04051aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:17.848709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1fcf1-0115-4670-9c9e-332c04051aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:17.956578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-7ac1fcf1-0115-4670-9c9e-332c04051aa6 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:17.958437 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7ac1fcf1-0115-4670-9c9e-332c04051aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:18.171643 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1268625-14fb-481c-9b3c-e05486612dfd None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:18.176849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 840/1698] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:18.184239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1fcf1-0115-4670-9c9e-332c04051aa6 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:18.192818 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452873ab-435e-4bf5-b419-f0a5a30ff799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:18.195889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452873ab-435e-4bf5-b419-f0a5a30ff799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:18.196333 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452873ab-435e-4bf5-b419-f0a5a30ff799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:18.196775 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452873ab-435e-4bf5-b419-f0a5a30ff799 admin admin] PATH_INFO: `/v3/projects/0f0f0365fdce40ae865b5babfacea405` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:18.197752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7ac1fcf1-0115-4670-9c9e-332c04051aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:18.205338 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452873ab-435e-4bf5-b419-f0a5a30ff799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f0f0365fdce40ae865b5babfacea405)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:18.210006 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452873ab-435e-4bf5-b419-f0a5a30ff799 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:18.255856 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1fcf1-0115-4670-9c9e-332c04051aa6 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:18.280741 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dea3a7-7d50-4246-9185-1bdeaf0705ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:18.281503 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dea3a7-7d50-4246-9185-1bdeaf0705ff service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:18.281887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dea3a7-7d50-4246-9185-1bdeaf0705ff service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:18.282285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dea3a7-7d50-4246-9185-1bdeaf0705ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:18.318010 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dea3a7-7d50-4246-9185-1bdeaf0705ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:18.320947 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dea3a7-7d50-4246-9185-1bdeaf0705ff service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:18.347249 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:18.357326 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 841/1699] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:18.360663 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5837d-df7d-4a78-b188-a6af47c53c32 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:18.360944 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5837d-df7d-4a78-b188-a6af47c53c32 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:18.361218 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b5837d-df7d-4a78-b188-a6af47c53c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:18.368714 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-452873ab-435e-4bf5-b419-f0a5a30ff799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f0f0365fdce40ae865b5babfacea405', 'request_id': 'req-452873ab-435e-4bf5-b419-f0a5a30ff799'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:18.391668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-452873ab-435e-4bf5-b419-f0a5a30ff799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f0f0365fdce40ae865b5babfacea405', 'request_id': 'req-452873ab-435e-4bf5-b419-f0a5a30ff799'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:18.405697 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-452873ab-435e-4bf5-b419-f0a5a30ff799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f0f0365fdce40ae865b5babfacea405', 'request_id': 'req-452873ab-435e-4bf5-b419-f0a5a30ff799'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:18.429023 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-f9b5837d-df7d-4a78-b188-a6af47c53c32 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:18.431482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f9b5837d-df7d-4a78-b188-a6af47c53c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:18.446876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-452873ab-435e-4bf5-b419-f0a5a30ff799 admin admin] The token cache is being invalidate because project 0f0f0365fdce40ae865b5babfacea405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:18.447412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-452873ab-435e-4bf5-b419-f0a5a30ff799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:18.448936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-452873ab-435e-4bf5-b419-f0a5a30ff799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:18.451582 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 859/1700] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:17 2022] DELETE /identity/v3/projects/0f0f0365fdce40ae865b5babfacea405 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:18.458917 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da3515-c896-4947-ab0d-ae7788eeab06 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:18.459217 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da3515-c896-4947-ab0d-ae7788eeab06 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:18.459493 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da3515-c896-4947-ab0d-ae7788eeab06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:18.488684 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:18.535239 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-d1da3515-c896-4947-ab0d-ae7788eeab06 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:18.540583 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d1da3515-c896-4947-ab0d-ae7788eeab06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:18.681151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 842/1701] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:18.681151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6a080-632e-4a78-89f8-82235fd2c35d None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:18.681151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6a080-632e-4a78-89f8-82235fd2c35d None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:18.681151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6a080-632e-4a78-89f8-82235fd2c35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:18.729655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 860/1702] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:18.731688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da3515-c896-4947-ab0d-ae7788eeab06 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:18.745404 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da3515-c896-4947-ab0d-ae7788eeab06 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:18.755678 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b695e1f-3b10-4657-81fa-a5d6f2105bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:18.758243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b695e1f-3b10-4657-81fa-a5d6f2105bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:18.758802 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b695e1f-3b10-4657-81fa-a5d6f2105bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:18.759672 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b695e1f-3b10-4657-81fa-a5d6f2105bd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:18.761093 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b695e1f-3b10-4657-81fa-a5d6f2105bd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:18.761640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b695e1f-3b10-4657-81fa-a5d6f2105bd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:18.764335 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b695e1f-3b10-4657-81fa-a5d6f2105bd3 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:18.775472 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-7cb6a080-632e-4a78-89f8-82235fd2c35d None None] MFA Rules not processed for user `0addaa15e4c844a5acd84c3a5e3869ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:18.777276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7cb6a080-632e-4a78-89f8-82235fd2c35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:18.791411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 861/1703] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:18.793043 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b25e8e-8da3-4c26-9740-711428936033 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:18.793988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b25e8e-8da3-4c26-9740-711428936033 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:18.794647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b25e8e-8da3-4c26-9740-711428936033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:18.831714 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-c8b25e8e-8da3-4c26-9740-711428936033 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:18.837470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-c8b25e8e-8da3-4c26-9740-711428936033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:18.883115 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 862/1704] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:18.884231 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b25e8e-8da3-4c26-9740-711428936033 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:18.900738 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b25e8e-8da3-4c26-9740-711428936033 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:18.906542 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a7094-ae2e-4f81-94b2-2df76d423cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:18.907352 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a7094-ae2e-4f81-94b2-2df76d423cd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:18.907772 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a7094-ae2e-4f81-94b2-2df76d423cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:18.908332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a7094-ae2e-4f81-94b2-2df76d423cd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:18.909236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a7094-ae2e-4f81-94b2-2df76d423cd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:18.918778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a7094-ae2e-4f81-94b2-2df76d423cd9 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:18.971808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 843/1705] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:18.978512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6a080-632e-4a78-89f8-82235fd2c35d None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:19.002435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6a080-632e-4a78-89f8-82235fd2c35d None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:19.018020 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 863/1706] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:18 2022] POST /identity/v3/projects => generated 393 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:19.019600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327aca6-3e38-483b-afcf-6e41d421a5da None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:19.019888 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327aca6-3e38-483b-afcf-6e41d421a5da None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:19.020281 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327aca6-3e38-483b-afcf-6e41d421a5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:19.022989 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904877f-dc93-4729-8bff-b092d664012f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:19.023956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904877f-dc93-4729-8bff-b092d664012f admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:19.024080 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904877f-dc93-4729-8bff-b092d664012f admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:19.024339 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904877f-dc93-4729-8bff-b092d664012f admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:19.027059 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904877f-dc93-4729-8bff-b092d664012f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:19.027059 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904877f-dc93-4729-8bff-b092d664012f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:19.027244 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904877f-dc93-4729-8bff-b092d664012f admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:19.051381 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 844/1707] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:19.051584 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-b327aca6-3e38-483b-afcf-6e41d421a5da None None] MFA Rules not processed for user `0addaa15e4c844a5acd84c3a5e3869ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:19.052855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b327aca6-3e38-483b-afcf-6e41d421a5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:19.055454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904877f-dc93-4729-8bff-b092d664012f admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:19.077740 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904877f-dc93-4729-8bff-b092d664012f admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:19.166757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 864/1708] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:19.168032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327aca6-3e38-483b-afcf-6e41d421a5da None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:19.237460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7d867-2dd6-4681-bc8c-c5dc43118d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:19.239005 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7d867-2dd6-4681-bc8c-c5dc43118d19 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:19.239477 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7d867-2dd6-4681-bc8c-c5dc43118d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:19.240667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7d867-2dd6-4681-bc8c-c5dc43118d19 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:19.241472 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7d867-2dd6-4681-bc8c-c5dc43118d19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:19.245960 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7d867-2dd6-4681-bc8c-c5dc43118d19 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:19.249670 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:19.280483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 845/1709] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:19 2022] POST /identity/v3/users => generated 297 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:19.285640 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7d867-2dd6-4681-bc8c-c5dc43118d19 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:19.289863 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-bca7d867-2dd6-4681-bc8c-c5dc43118d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:19.361134 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327aca6-3e38-483b-afcf-6e41d421a5da None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:19.373672 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9b364-015d-4d8d-9060-dd3150550dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:19.374303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9b364-015d-4d8d-9060-dd3150550dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:19.374586 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9b364-015d-4d8d-9060-dd3150550dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:19.375127 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:19.375747 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9b364-015d-4d8d-9060-dd3150550dd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:19.376234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca9b364-015d-4d8d-9060-dd3150550dd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:19.381420 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca9b364-015d-4d8d-9060-dd3150550dd1 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:19.451233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 865/1710] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:19 2022] POST /identity/v3/projects => generated 363 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:19.455329 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55fe25-a24a-4604-b39b-058c8f5b2d23 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:19.455618 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55fe25-a24a-4604-b39b-058c8f5b2d23 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:19.455898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55fe25-a24a-4604-b39b-058c8f5b2d23 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:19.462662 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 866/1711] 10.222.0.26 () {54 vars in 912 bytes} [Thu Sep 22 14:58:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:19.463778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55fe25-a24a-4604-b39b-058c8f5b2d23 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:19.471647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ca55fe25-a24a-4604-b39b-058c8f5b2d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:19.488598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7d867-2dd6-4681-bc8c-c5dc43118d19 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:19.634598 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55fe25-a24a-4604-b39b-058c8f5b2d23 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:19.640173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:19.640963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:19.641349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:19.642148 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f admin admin] PATH_INFO: `/v3/projects/88d7db84f4364c589668c55826cce241` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:19.643033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f admin admin] RBAC: Authorizing `identity:delete_project(project_id=88d7db84f4364c589668c55826cce241)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:19.645208 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:19.739996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049319e1-a2f9-42df-80bf-8dea9d6635a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:19.743345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049319e1-a2f9-42df-80bf-8dea9d6635a7 service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:19.743626 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049319e1-a2f9-42df-80bf-8dea9d6635a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:19.744837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049319e1-a2f9-42df-80bf-8dea9d6635a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:19.768772 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88d7db84f4364c589668c55826cce241', 'request_id': 'req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:19.782457 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88d7db84f4364c589668c55826cce241', 'request_id': 'req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:19.795337 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88d7db84f4364c589668c55826cce241', 'request_id': 'req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:19.830317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f admin admin] The token cache is being invalidate because project 88d7db84f4364c589668c55826cce241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:19.830317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:19.830317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:19.836224 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 867/1712] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:19 2022] DELETE /identity/v3/projects/88d7db84f4364c589668c55826cce241 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:19.838152 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:19.838683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:19.926921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049319e1-a2f9-42df-80bf-8dea9d6635a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:19.928553 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049319e1-a2f9-42df-80bf-8dea9d6635a7 service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:19.965500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:19.970230 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 846/1713] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 685 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:19.971654 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fb981-64e0-40bd-86bb-4d86d590ba9b None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:19.971765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fb981-64e0-40bd-86bb-4d86d590ba9b None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:19.972284 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fb981-64e0-40bd-86bb-4d86d590ba9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.021158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d85f2-2da6-4ba5-b3f9-adc86215231f admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:20.046805 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e33709-fb5f-4d60-9f59-49a79c910b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:20.047458 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e33709-fb5f-4d60-9f59-49a79c910b95 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.047742 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e33709-fb5f-4d60-9f59-49a79c910b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.048016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e33709-fb5f-4d60-9f59-49a79c910b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.048831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e33709-fb5f-4d60-9f59-49a79c910b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:20.050507 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e33709-fb5f-4d60-9f59-49a79c910b95 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:20.067865 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-9d5fb981-64e0-40bd-86bb-4d86d590ba9b None None] MFA Rules not processed for user `23d683cdc90041b5b29767fa2e2107e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:20.069252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-9d5fb981-64e0-40bd-86bb-4d86d590ba9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:20.072289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 868/1714] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:20.077896 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e33709-fb5f-4d60-9f59-49a79c910b95 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:20.080926 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-f3e33709-fb5f-4d60-9f59-49a79c910b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:20.121012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e33709-fb5f-4d60-9f59-49a79c910b95 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:20.133388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2b636-3047-468a-99aa-5327bcd4e26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:20.134147 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2b636-3047-468a-99aa-5327bcd4e26a admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.134539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2b636-3047-468a-99aa-5327bcd4e26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.137370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2b636-3047-468a-99aa-5327bcd4e26a admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.138162 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2b636-3047-468a-99aa-5327bcd4e26a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:20.141949 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2b636-3047-468a-99aa-5327bcd4e26a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:20.206133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 869/1715] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:20 2022] POST /identity/v3/users => generated 282 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:20.208352 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491d1e-f7db-423d-b77f-c15810457cdc None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.213434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491d1e-f7db-423d-b77f-c15810457cdc None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.213931 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62491d1e-f7db-423d-b77f-c15810457cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.259434 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:20.308569 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 847/1716] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:20.308569 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb550a-945f-46e5-9051-571f80db90ac None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.308569 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb550a-945f-46e5-9051-571f80db90ac None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.308569 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb550a-945f-46e5-9051-571f80db90ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.323336 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-62491d1e-f7db-423d-b77f-c15810457cdc None None] MFA Rules not processed for user `f77bbd8de87042aab3674db366635cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:20.326178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-62491d1e-f7db-423d-b77f-c15810457cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:20.384832 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-2cdb550a-945f-46e5-9051-571f80db90ac None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:20.387423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2cdb550a-945f-46e5-9051-571f80db90ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:20.440979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 848/1717] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:20.446547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb550a-945f-46e5-9051-571f80db90ac None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:20.480908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb550a-945f-46e5-9051-571f80db90ac None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:20.492819 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41274a66-54d7-42b6-99d4-f7a1739a4f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:20.494262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41274a66-54d7-42b6-99d4-f7a1739a4f46 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.495138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41274a66-54d7-42b6-99d4-f7a1739a4f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.495568 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41274a66-54d7-42b6-99d4-f7a1739a4f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.498843 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41274a66-54d7-42b6-99d4-f7a1739a4f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:20.503254 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41274a66-54d7-42b6-99d4-f7a1739a4f46 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:20.527574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 849/1718] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:20.531175 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41274a66-54d7-42b6-99d4-f7a1739a4f46 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:20.547439 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 870/1719] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Sep 22 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:20.549064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7193273-272d-48a1-91e1-2567d2020005 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.549583 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7193273-272d-48a1-91e1-2567d2020005 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.549978 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7193273-272d-48a1-91e1-2567d2020005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.557351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41274a66-54d7-42b6-99d4-f7a1739a4f46 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:20.568403 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812dda65-7140-4f0e-ad82-f2db9ef1c7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:20.569410 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dda65-7140-4f0e-ad82-f2db9ef1c7fe admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.571154 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dda65-7140-4f0e-ad82-f2db9ef1c7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.571832 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812dda65-7140-4f0e-ad82-f2db9ef1c7fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.572761 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812dda65-7140-4f0e-ad82-f2db9ef1c7fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:20.582874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-e7193273-272d-48a1-91e1-2567d2020005 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:20.584212 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e7193273-272d-48a1-91e1-2567d2020005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:20.590758 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812dda65-7140-4f0e-ad82-f2db9ef1c7fe admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:20.594223 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 871/1720] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:20.595398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7193273-272d-48a1-91e1-2567d2020005 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:20.608065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7193273-272d-48a1-91e1-2567d2020005 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:20.613245 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 850/1721] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:20.614282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812dda65-7140-4f0e-ad82-f2db9ef1c7fe admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:20.614941 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95594f-63ae-41f3-9938-77b16d626375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:20.615881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95594f-63ae-41f3-9938-77b16d626375 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.616111 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-812dda65-7140-4f0e-ad82-f2db9ef1c7fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:20.616356 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95594f-63ae-41f3-9938-77b16d626375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.617251 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f95594f-63ae-41f3-9938-77b16d626375 admin admin] PATH_INFO: `/v3/projects/9a99dffe9d0c406fb4fbd51a8406f7f2/users/debc8889dd0e4979a1d7cce8dafe48cc/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.642784 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95594f-63ae-41f3-9938-77b16d626375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a99dffe9d0c406fb4fbd51a8406f7f2, user_id=debc8889dd0e4979a1d7cce8dafe48cc, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:20.649460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812dda65-7140-4f0e-ad82-f2db9ef1c7fe admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:20.649747 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f95594f-63ae-41f3-9938-77b16d626375 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:20.654954 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2085a60-d5cc-4f42-a298-fa6dd68564e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:20.656176 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2085a60-d5cc-4f42-a298-fa6dd68564e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.656391 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2085a60-d5cc-4f42-a298-fa6dd68564e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.656627 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2085a60-d5cc-4f42-a298-fa6dd68564e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.681482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2085a60-d5cc-4f42-a298-fa6dd68564e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:20.681482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2085a60-d5cc-4f42-a298-fa6dd68564e5 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:20.682389 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 872/1722] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:20 2022] PUT /identity/v3/projects/9a99dffe9d0c406fb4fbd51a8406f7f2/users/debc8889dd0e4979a1d7cce8dafe48cc/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:20.683696 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95594f-63ae-41f3-9938-77b16d626375 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:20.685870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 851/1723] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:20.687201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2085a60-d5cc-4f42-a298-fa6dd68564e5 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:20.712280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2085a60-d5cc-4f42-a298-fa6dd68564e5 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:20.842622 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433f7b9-ccbb-4423-90ab-c2b4cb811cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:20.845140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433f7b9-ccbb-4423-90ab-c2b4cb811cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.849450 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433f7b9-ccbb-4423-90ab-c2b4cb811cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.853254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f95594f-63ae-41f3-9938-77b16d626375 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:20.853793 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433f7b9-ccbb-4423-90ab-c2b4cb811cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.855091 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433f7b9-ccbb-4423-90ab-c2b4cb811cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:20.862562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433f7b9-ccbb-4423-90ab-c2b4cb811cf7 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:20.867611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5412f1-668f-46ed-9572-b49d5e28a3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:20.868418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5412f1-668f-46ed-9572-b49d5e28a3e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.869307 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5412f1-668f-46ed-9572-b49d5e28a3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.870509 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5412f1-668f-46ed-9572-b49d5e28a3e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.870509 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5412f1-668f-46ed-9572-b49d5e28a3e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:20.870955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5412f1-668f-46ed-9572-b49d5e28a3e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:20.872983 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5412f1-668f-46ed-9572-b49d5e28a3e6 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:20.896024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 873/1724] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:20.896024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76eb56-32e5-4d41-9d0b-d881d0e70413 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.896024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76eb56-32e5-4d41-9d0b-d881d0e70413 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.896024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76eb56-32e5-4d41-9d0b-d881d0e70413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.904587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 852/1725] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:20.910384 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f19f2-fd6a-4413-af0c-a2b474e449bf None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.912017 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f19f2-fd6a-4413-af0c-a2b474e449bf None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.912436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f19f2-fd6a-4413-af0c-a2b474e449bf None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.917371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 853/1726] 10.222.0.26 () {54 vars in 912 bytes} [Thu Sep 22 14:58:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:20.918221 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f19f2-fd6a-4413-af0c-a2b474e449bf None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:20.939145 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-6d76eb56-32e5-4d41-9d0b-d881d0e70413 None None] MFA Rules not processed for user `debc8889dd0e4979a1d7cce8dafe48cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:20.942232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-6d76eb56-32e5-4d41-9d0b-d881d0e70413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:20.945111 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f19f2-fd6a-4413-af0c-a2b474e449bf None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:20.959408 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:20.962124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2feb6a-6c04-4a43-8923-f99e4e57054e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:20.963487 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2feb6a-6c04-4a43-8923-f99e4e57054e admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:20.963714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2feb6a-6c04-4a43-8923-f99e4e57054e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:20.964474 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2feb6a-6c04-4a43-8923-f99e4e57054e admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:20.964632 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2feb6a-6c04-4a43-8923-f99e4e57054e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:20.970448 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2feb6a-6c04-4a43-8923-f99e4e57054e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:21.012432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 854/1727] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:20 2022] POST /identity/v3/projects => generated 365 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:21.015872 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2feb6a-6c04-4a43-8923-f99e4e57054e admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:21.117432 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 874/1728] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:21.119299 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7f33b-fc9c-479c-b71b-8c98635e4e28 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:21.119736 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7f33b-fc9c-479c-b71b-8c98635e4e28 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:21.120141 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7f33b-fc9c-479c-b71b-8c98635e4e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:21.157504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-b8f7f33b-fc9c-479c-b71b-8c98635e4e28 None None] MFA Rules not processed for user `f77bbd8de87042aab3674db366635cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:21.159180 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b8f7f33b-fc9c-479c-b71b-8c98635e4e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:21.168290 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2feb6a-6c04-4a43-8923-f99e4e57054e admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:21.180826 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be2e0e-2833-4ebf-b6e2-e92be9a32d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:21.182652 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be2e0e-2833-4ebf-b6e2-e92be9a32d56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:21.182994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be2e0e-2833-4ebf-b6e2-e92be9a32d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:21.183270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be2e0e-2833-4ebf-b6e2-e92be9a32d56 admin admin] PATH_INFO: `/v3/projects/78e15d310c6549579aa79a554475e041/users/2eb470b36e77498491d34abeef7e4314/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:21.221650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be2e0e-2833-4ebf-b6e2-e92be9a32d56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78e15d310c6549579aa79a554475e041, user_id=2eb470b36e77498491d34abeef7e4314, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:21.226905 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be2e0e-2833-4ebf-b6e2-e92be9a32d56 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:21.256187 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 875/1729] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Sep 22 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:21.256887 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7f33b-fc9c-479c-b71b-8c98635e4e28 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:21.260467 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 855/1730] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:21 2022] PUT /identity/v3/projects/78e15d310c6549579aa79a554475e041/users/2eb470b36e77498491d34abeef7e4314/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:21.261949 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be2e0e-2833-4ebf-b6e2-e92be9a32d56 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:21.270036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7f33b-fc9c-479c-b71b-8c98635e4e28 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:21.284657 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:21.416335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be2e0e-2833-4ebf-b6e2-e92be9a32d56 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:21.425418 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611d3fa-53d2-4e9a-b912-5f55d3941c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:21.434429 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611d3fa-53d2-4e9a-b912-5f55d3941c8f admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:21.435078 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611d3fa-53d2-4e9a-b912-5f55d3941c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:21.435538 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611d3fa-53d2-4e9a-b912-5f55d3941c8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:21.436894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611d3fa-53d2-4e9a-b912-5f55d3941c8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:21.440338 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611d3fa-53d2-4e9a-b912-5f55d3941c8f admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:21.446295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc070c6-e973-49cb-9c94-e1906fbce994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:21.447557 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc070c6-e973-49cb-9c94-e1906fbce994 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:21.448152 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc070c6-e973-49cb-9c94-e1906fbce994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:21.448550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc070c6-e973-49cb-9c94-e1906fbce994 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:21.449903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc070c6-e973-49cb-9c94-e1906fbce994 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:21.456182 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc070c6-e973-49cb-9c94-e1906fbce994 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:21.498883 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 856/1731] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:21 2022] POST /identity/v3/projects => generated 391 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:21.501301 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611d3fa-53d2-4e9a-b912-5f55d3941c8f admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:21.510530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 876/1732] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:21 2022] POST /identity/v3/users => generated 283 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:21.514357 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a606552-07f2-40a9-9f24-6ed1cd3b57d2 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:21.514961 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a606552-07f2-40a9-9f24-6ed1cd3b57d2 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:21.515749 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a606552-07f2-40a9-9f24-6ed1cd3b57d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:21.609895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-4a606552-07f2-40a9-9f24-6ed1cd3b57d2 None None] MFA Rules not processed for user `2eb470b36e77498491d34abeef7e4314`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:21.621702 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-4a606552-07f2-40a9-9f24-6ed1cd3b57d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:21.665443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611d3fa-53d2-4e9a-b912-5f55d3941c8f admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:21.678326 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4e747-c356-4530-9f2c-9b0d5b72d465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:21.678845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4e747-c356-4530-9f2c-9b0d5b72d465 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:21.679168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4e747-c356-4530-9f2c-9b0d5b72d465 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:21.684090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4e747-c356-4530-9f2c-9b0d5b72d465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:21.704127 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4e747-c356-4530-9f2c-9b0d5b72d465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:21.705926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4e747-c356-4530-9f2c-9b0d5b72d465 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:21.716298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:21.728168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 857/1733] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:21.728906 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4e747-c356-4530-9f2c-9b0d5b72d465 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:21.731848 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1ac4e747-c356-4530-9f2c-9b0d5b72d465 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:21.801775 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 877/1734] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:21.802818 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a606552-07f2-40a9-9f24-6ed1cd3b57d2 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:21.937469 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4e747-c356-4530-9f2c-9b0d5b72d465 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:21.960703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26870f7d-dd65-47dc-a750-7a6c1b94721f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:21.961355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26870f7d-dd65-47dc-a750-7a6c1b94721f service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:21.963895 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26870f7d-dd65-47dc-a750-7a6c1b94721f service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:21.964174 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26870f7d-dd65-47dc-a750-7a6c1b94721f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:22.030245 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a606552-07f2-40a9-9f24-6ed1cd3b57d2 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:22.039667 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b0181-fd59-4a13-a877-37c39815c95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:22.040477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b0181-fd59-4a13-a877-37c39815c95c admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:22.040856 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b0181-fd59-4a13-a877-37c39815c95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:22.041820 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b0181-fd59-4a13-a877-37c39815c95c admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:22.044248 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b0181-fd59-4a13-a877-37c39815c95c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:22.047713 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b0181-fd59-4a13-a877-37c39815c95c admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:22.080895 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:22.118199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 878/1735] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:21 2022] POST /identity/v3/users => generated 302 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:22.119617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b0181-fd59-4a13-a877-37c39815c95c admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:22.130259 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26870f7d-dd65-47dc-a750-7a6c1b94721f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:22.151215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26870f7d-dd65-47dc-a750-7a6c1b94721f service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:22.153199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b0181-fd59-4a13-a877-37c39815c95c admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:22.161843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fa430-8c3b-423f-94f7-172a8f0e8b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:22.162498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fa430-8c3b-423f-94f7-172a8f0e8b4b admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:22.162810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fa430-8c3b-423f-94f7-172a8f0e8b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:22.163090 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fa430-8c3b-423f-94f7-172a8f0e8b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:22.163786 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084fa430-8c3b-423f-94f7-172a8f0e8b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:22.165796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 858/1736] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:22.166845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26870f7d-dd65-47dc-a750-7a6c1b94721f service nova] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:22.168418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084fa430-8c3b-423f-94f7-172a8f0e8b4b admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:22.168551 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-26870f7d-dd65-47dc-a750-7a6c1b94721f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:22.213699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 879/1737] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:22.221299 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fa430-8c3b-423f-94f7-172a8f0e8b4b admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:22.235729 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fa430-8c3b-423f-94f7-172a8f0e8b4b admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:22.240925 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397559f-eaca-4693-8800-395c717d77b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:22.241543 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397559f-eaca-4693-8800-395c717d77b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:22.242321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397559f-eaca-4693-8800-395c717d77b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:22.243042 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397559f-eaca-4693-8800-395c717d77b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:22.243567 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397559f-eaca-4693-8800-395c717d77b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:22.247459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397559f-eaca-4693-8800-395c717d77b8 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:22.256951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26870f7d-dd65-47dc-a750-7a6c1b94721f service nova] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:22.264270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250485d-3f13-4d60-a95a-9de5e61b4e82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b6b34b4ee5492fb1f0699716e5118b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:22.266881 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250485d-3f13-4d60-a95a-9de5e61b4e82 service placement] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:22.267254 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250485d-3f13-4d60-a95a-9de5e61b4e82 service placement] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:22.267599 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250485d-3f13-4d60-a95a-9de5e61b4e82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:22.272690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1250485d-3f13-4d60-a95a-9de5e61b4e82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:22.306654 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1250485d-3f13-4d60-a95a-9de5e61b4e82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:22.310562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1250485d-3f13-4d60-a95a-9de5e61b4e82 service placement] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:22.313787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 880/1738] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:22 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:22.318240 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397559f-eaca-4693-8800-395c717d77b8 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:22.339236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 859/1739] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:22.341641 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250485d-3f13-4d60-a95a-9de5e61b4e82 service placement] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:22.356880 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250485d-3f13-4d60-a95a-9de5e61b4e82 service placement] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:22.582061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51bb38-1415-4d98-850c-96cce58dfd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:22.582921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51bb38-1415-4d98-850c-96cce58dfd56 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:22.586191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51bb38-1415-4d98-850c-96cce58dfd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:22.586605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51bb38-1415-4d98-850c-96cce58dfd56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:22.590132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51bb38-1415-4d98-850c-96cce58dfd56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:22.594261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397559f-eaca-4693-8800-395c717d77b8 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:22.594838 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51bb38-1415-4d98-850c-96cce58dfd56 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:22.615559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a66808-42e4-41e8-a44c-c945293790d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:22.616262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a66808-42e4-41e8-a44c-c945293790d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:22.616589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a66808-42e4-41e8-a44c-c945293790d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:22.616866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a66808-42e4-41e8-a44c-c945293790d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:22.617532 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a66808-42e4-41e8-a44c-c945293790d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:22.630259 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a66808-42e4-41e8-a44c-c945293790d3 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:22.637496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 860/1740] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:22.643539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51bb38-1415-4d98-850c-96cce58dfd56 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:22.683024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51bb38-1415-4d98-850c-96cce58dfd56 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:22.695781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 881/1741] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:22.698432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f698d4-013f-439d-b9f2-05f04b3a7026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:22.700261 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f698d4-013f-439d-b9f2-05f04b3a7026 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:22.705323 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f698d4-013f-439d-b9f2-05f04b3a7026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:22.705904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f698d4-013f-439d-b9f2-05f04b3a7026 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:22.706840 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f698d4-013f-439d-b9f2-05f04b3a7026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:22.706965 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d06cc2-987c-4d5b-95d4-a95eeb486920 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:22.707199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d06cc2-987c-4d5b-95d4-a95eeb486920 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:22.707881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d06cc2-987c-4d5b-95d4-a95eeb486920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:22.715548 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f698d4-013f-439d-b9f2-05f04b3a7026 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:22.780035 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 861/1742] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:22 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:22.781169 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f698d4-013f-439d-b9f2-05f04b3a7026 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:22.803330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f698d4-013f-439d-b9f2-05f04b3a7026 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:22.820709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5994b24-3a22-44e9-bb51-418922ea339e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:22.821508 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5994b24-3a22-44e9-bb51-418922ea339e admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:22.821890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5994b24-3a22-44e9-bb51-418922ea339e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:22.822289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5994b24-3a22-44e9-bb51-418922ea339e admin admin] PATH_INFO: `/v3/projects/ffb6e23c854e4ad394abb04963f2d896/users/92d3c59f36be4b46859bb29397fd728c/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:22.838050 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-57d06cc2-987c-4d5b-95d4-a95eeb486920 None None] MFA Rules not processed for user `f298f0c6a9e44efba2a1e2fc648a0f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:22.849422 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-57d06cc2-987c-4d5b-95d4-a95eeb486920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:22.885130 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5994b24-3a22-44e9-bb51-418922ea339e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb6e23c854e4ad394abb04963f2d896, user_id=92d3c59f36be4b46859bb29397fd728c, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:22.898837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5994b24-3a22-44e9-bb51-418922ea339e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:22.941486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 862/1743] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:22 2022] PUT /identity/v3/projects/ffb6e23c854e4ad394abb04963f2d896/users/92d3c59f36be4b46859bb29397fd728c/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:22.945380 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5994b24-3a22-44e9-bb51-418922ea339e admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:22.996630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:23.032405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:23.114235 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 882/1744] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:23.128921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ae7bb-ba5e-4901-86f6-851c0726c91e None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:23.136211 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ae7bb-ba5e-4901-86f6-851c0726c91e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:23.138466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ae7bb-ba5e-4901-86f6-851c0726c91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:23.197898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-cb2ae7bb-ba5e-4901-86f6-851c0726c91e None None] MFA Rules not processed for user `038365d6ba7041f4af386767d98ed709`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:23.199433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-cb2ae7bb-ba5e-4901-86f6-851c0726c91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:23.247317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5994b24-3a22-44e9-bb51-418922ea339e admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:23.279930 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281cab14-c4a1-44a0-9c18-e03837b530a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:23.280635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cab14-c4a1-44a0-9c18-e03837b530a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:23.280982 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cab14-c4a1-44a0-9c18-e03837b530a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:23.281177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cab14-c4a1-44a0-9c18-e03837b530a7 admin admin] PATH_INFO: `/v3/projects/3da5c029dcf64920aa67c56b1b007806/users/09f3966f30494eacbad130db240b86eb/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:23.359209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281cab14-c4a1-44a0-9c18-e03837b530a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3da5c029dcf64920aa67c56b1b007806, user_id=09f3966f30494eacbad130db240b86eb, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:23.361916 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281cab14-c4a1-44a0-9c18-e03837b530a7 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:23.372321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 883/1745] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:23.373512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ae7bb-ba5e-4901-86f6-851c0726c91e None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:23.400020 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ae7bb-ba5e-4901-86f6-851c0726c91e None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:23.411007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc1790-e917-4905-b5a0-e0a01fe77252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:23.412173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc1790-e917-4905-b5a0-e0a01fe77252 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:23.412540 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc1790-e917-4905-b5a0-e0a01fe77252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:23.412923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc1790-e917-4905-b5a0-e0a01fe77252 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:23.413784 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc1790-e917-4905-b5a0-e0a01fe77252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:23.415549 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc1790-e917-4905-b5a0-e0a01fe77252 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:23.439542 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 863/1746] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:22 2022] PUT /identity/v3/projects/3da5c029dcf64920aa67c56b1b007806/users/09f3966f30494eacbad130db240b86eb/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:23.452592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75fab8-a57a-47b8-94dd-78cb7925ad9c None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:23.452592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75fab8-a57a-47b8-94dd-78cb7925ad9c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:23.452592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75fab8-a57a-47b8-94dd-78cb7925ad9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:23.479257 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 884/1747] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:23.480105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc1790-e917-4905-b5a0-e0a01fe77252 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:23.529082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc1790-e917-4905-b5a0-e0a01fe77252 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:23.587874 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-0a75fab8-a57a-47b8-94dd-78cb7925ad9c None None] MFA Rules not processed for user `92d3c59f36be4b46859bb29397fd728c`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:23.591470 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0a75fab8-a57a-47b8-94dd-78cb7925ad9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:23.783536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9d5d0-c8ce-4f52-8e55-32200659510e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:23.784179 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9d5d0-c8ce-4f52-8e55-32200659510e admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:23.784458 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9d5d0-c8ce-4f52-8e55-32200659510e admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:23.784745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9d5d0-c8ce-4f52-8e55-32200659510e admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:23.785541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9d5d0-c8ce-4f52-8e55-32200659510e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:23.787067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9d5d0-c8ce-4f52-8e55-32200659510e admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:23.848754 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 885/1748] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:23.848754 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9d5d0-c8ce-4f52-8e55-32200659510e admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:23.908258 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9d5d0-c8ce-4f52-8e55-32200659510e admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:23.927875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 864/1749] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 488 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:23.930297 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75fab8-a57a-47b8-94dd-78cb7925ad9c None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:23.934657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568ee63-df1f-4c5c-b93f-12335bc46adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:23.935382 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568ee63-df1f-4c5c-b93f-12335bc46adb admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:23.935663 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568ee63-df1f-4c5c-b93f-12335bc46adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:23.935937 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568ee63-df1f-4c5c-b93f-12335bc46adb admin admin] PATH_INFO: `/v3/projects/f92e28ecccf748fbb0dd0fb5761e642d/users/c182d62b14ef4d9db495cc89844c1050/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:23.983398 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75fab8-a57a-47b8-94dd-78cb7925ad9c None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:23.997477 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec9261-7d7f-433d-9adb-8fb6c5adef02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:23.998176 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec9261-7d7f-433d-9adb-8fb6c5adef02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:23.998455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec9261-7d7f-433d-9adb-8fb6c5adef02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:23.998766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec9261-7d7f-433d-9adb-8fb6c5adef02 admin admin] PATH_INFO: `/v3/projects/3da5c029dcf64920aa67c56b1b007806/users/09f3966f30494eacbad130db240b86eb/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:24.003749 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568ee63-df1f-4c5c-b93f-12335bc46adb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f92e28ecccf748fbb0dd0fb5761e642d, user_id=c182d62b14ef4d9db495cc89844c1050, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:24.006104 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec9261-7d7f-433d-9adb-8fb6c5adef02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3da5c029dcf64920aa67c56b1b007806, user_id=09f3966f30494eacbad130db240b86eb, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:24.013446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568ee63-df1f-4c5c-b93f-12335bc46adb admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:24.016685 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec9261-7d7f-433d-9adb-8fb6c5adef02 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:24.062482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 865/1750] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:23 2022] PUT /identity/v3/projects/3da5c029dcf64920aa67c56b1b007806/users/09f3966f30494eacbad130db240b86eb/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:24.062482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec9261-7d7f-433d-9adb-8fb6c5adef02 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:24.062482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-faec9261-7d7f-433d-9adb-8fb6c5adef02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:24.066709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 886/1751] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:23 2022] PUT /identity/v3/projects/f92e28ecccf748fbb0dd0fb5761e642d/users/c182d62b14ef4d9db495cc89844c1050/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:24.068767 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49147cc5-8cdf-4fee-b3d7-ff41d744028c None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:24.069050 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49147cc5-8cdf-4fee-b3d7-ff41d744028c None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:24.069323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49147cc5-8cdf-4fee-b3d7-ff41d744028c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:24.184543 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-49147cc5-8cdf-4fee-b3d7-ff41d744028c None None] MFA Rules not processed for user `09f3966f30494eacbad130db240b86eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:24.185817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-49147cc5-8cdf-4fee-b3d7-ff41d744028c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:24.306344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:24.328080 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec9261-7d7f-433d-9adb-8fb6c5adef02 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:24.436651 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 887/1752] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:24 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:24.437616 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49147cc5-8cdf-4fee-b3d7-ff41d744028c None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:24.547992 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:24.655825 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf22f6-7971-43d0-b306-69b9148528b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:24.656644 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf22f6-7971-43d0-b306-69b9148528b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:24.657478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf22f6-7971-43d0-b306-69b9148528b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:24.657787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf22f6-7971-43d0-b306-69b9148528b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:24.685439 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49147cc5-8cdf-4fee-b3d7-ff41d744028c None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:24.709652 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d09bd-f254-4890-82bc-eae5e1413c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:24.710967 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d09bd-f254-4890-82bc-eae5e1413c7d admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:24.711245 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d09bd-f254-4890-82bc-eae5e1413c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:24.711517 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d09bd-f254-4890-82bc-eae5e1413c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:24.712186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d09bd-f254-4890-82bc-eae5e1413c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:24.716595 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d09bd-f254-4890-82bc-eae5e1413c7d admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:24.759090 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 888/1753] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:24.759090 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d09bd-f254-4890-82bc-eae5e1413c7d admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:24.762428 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-df0d09bd-f254-4890-82bc-eae5e1413c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:24.809473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d09bd-f254-4890-82bc-eae5e1413c7d admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:24.831345 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b97bbda-67ca-4471-b67e-f3376dcd9fda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:24.831954 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b97bbda-67ca-4471-b67e-f3376dcd9fda service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:24.832237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b97bbda-67ca-4471-b67e-f3376dcd9fda service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:24.832516 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b97bbda-67ca-4471-b67e-f3376dcd9fda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:24.865211 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b97bbda-67ca-4471-b67e-f3376dcd9fda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:24.871839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b97bbda-67ca-4471-b67e-f3376dcd9fda service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:24.907906 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf22f6-7971-43d0-b306-69b9148528b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:24.910843 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf22f6-7971-43d0-b306-69b9148528b8 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:24.919346 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 889/1754] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:24 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:24.920323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b97bbda-67ca-4471-b67e-f3376dcd9fda service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:24.954960 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b97bbda-67ca-4471-b67e-f3376dcd9fda service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:24.961693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 866/1755] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 910 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:24.985197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938426a6-4221-4211-816b-2c54a74a9530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:24.986117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938426a6-4221-4211-816b-2c54a74a9530 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:24.986337 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938426a6-4221-4211-816b-2c54a74a9530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:24.986461 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938426a6-4221-4211-816b-2c54a74a9530 admin admin] PATH_INFO: `/v3/projects/f92e28ecccf748fbb0dd0fb5761e642d/users/c182d62b14ef4d9db495cc89844c1050/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:24.988653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938426a6-4221-4211-816b-2c54a74a9530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f92e28ecccf748fbb0dd0fb5761e642d, user_id=c182d62b14ef4d9db495cc89844c1050, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:24.991913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938426a6-4221-4211-816b-2c54a74a9530 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:25.052419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 890/1756] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:24 2022] PUT /identity/v3/projects/f92e28ecccf748fbb0dd0fb5761e642d/users/c182d62b14ef4d9db495cc89844c1050/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:25.056796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b8d5f-bdb6-492d-a657-1da5da2cc162 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:25.057290 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b8d5f-bdb6-492d-a657-1da5da2cc162 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:25.057735 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b8d5f-bdb6-492d-a657-1da5da2cc162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:25.115406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-523b8d5f-bdb6-492d-a657-1da5da2cc162 None None] MFA Rules not processed for user `c182d62b14ef4d9db495cc89844c1050`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:25.118571 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-523b8d5f-bdb6-492d-a657-1da5da2cc162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:25.204509 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938426a6-4221-4211-816b-2c54a74a9530 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:25.381960 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 867/1757] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:25.396639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938426a6-4221-4211-816b-2c54a74a9530 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:25.408544 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9649d3-205a-4dcf-96ac-1904129d94e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:25.417221 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9649d3-205a-4dcf-96ac-1904129d94e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:25.417562 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9649d3-205a-4dcf-96ac-1904129d94e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:25.418082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9649d3-205a-4dcf-96ac-1904129d94e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:25.528796 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:25.554719 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9649d3-205a-4dcf-96ac-1904129d94e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:25.556258 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9649d3-205a-4dcf-96ac-1904129d94e1 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:25.578307 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 891/1758] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:25 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:25.672252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b8d5f-bdb6-492d-a657-1da5da2cc162 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:25.685119 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b8d5f-bdb6-492d-a657-1da5da2cc162 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:25.702931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528d7d0-692a-441e-8f3f-e69186cf5a50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:25.703595 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528d7d0-692a-441e-8f3f-e69186cf5a50 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:25.703863 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528d7d0-692a-441e-8f3f-e69186cf5a50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:25.704136 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528d7d0-692a-441e-8f3f-e69186cf5a50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:25.742216 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528d7d0-692a-441e-8f3f-e69186cf5a50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:25.743727 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528d7d0-692a-441e-8f3f-e69186cf5a50 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:25.802957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 868/1759] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:26.060843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9649d3-205a-4dcf-96ac-1904129d94e1 service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:26.075622 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9649d3-205a-4dcf-96ac-1904129d94e1 service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:26.085719 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e899e71-9e3d-4fde-aa32-d8299d50cb10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:26.089146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e899e71-9e3d-4fde-aa32-d8299d50cb10 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:26.089589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e899e71-9e3d-4fde-aa32-d8299d50cb10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:26.089977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e899e71-9e3d-4fde-aa32-d8299d50cb10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:26.244941 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e899e71-9e3d-4fde-aa32-d8299d50cb10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:26.247628 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e899e71-9e3d-4fde-aa32-d8299d50cb10 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:26.287885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 892/1760] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:27.330660 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac779a63-ad70-4a49-9297-516763f6f1fb None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:27.331206 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac779a63-ad70-4a49-9297-516763f6f1fb None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:27.331296 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac779a63-ad70-4a49-9297-516763f6f1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:27.376901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-ac779a63-ad70-4a49-9297-516763f6f1fb None None] MFA Rules not processed for user `23d683cdc90041b5b29767fa2e2107e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:27.395191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-ac779a63-ad70-4a49-9297-516763f6f1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:27.533053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c00f13-d540-492f-bc1e-5a48ba079e79 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:27.533715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c00f13-d540-492f-bc1e-5a48ba079e79 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:27.533715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c00f13-d540-492f-bc1e-5a48ba079e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:27.603817 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 869/1761] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:27.677544 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-73c00f13-d540-492f-bc1e-5a48ba079e79 None None] MFA Rules not processed for user `6256407d9df648d5bbe33495cb9a8b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:27.679198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-73c00f13-d540-492f-bc1e-5a48ba079e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:28.015151 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 893/1762] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Sep 22 14:58:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:28.420903 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac779a63-ad70-4a49-9297-516763f6f1fb None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:28.423292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-ac779a63-ad70-4a49-9297-516763f6f1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:28.432198 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:28.692884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac779a63-ad70-4a49-9297-516763f6f1fb None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:28.708059 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1372822-be8e-49cf-acce-47d92c9c4198 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a265623ae734936a340c97df852e227', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:28.710989 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1372822-be8e-49cf-acce-47d92c9c4198 service swift] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:28.711814 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1372822-be8e-49cf-acce-47d92c9c4198 service swift] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:28.712770 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1372822-be8e-49cf-acce-47d92c9c4198 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:28.739752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1372822-be8e-49cf-acce-47d92c9c4198 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:28.741752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1372822-be8e-49cf-acce-47d92c9c4198 service swift] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:28.780312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 870/1763] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Sep 22 14:58:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 14:58:31.482650 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c00f13-d540-492f-bc1e-5a48ba079e79 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:31.609859 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c00f13-d540-492f-bc1e-5a48ba079e79 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:31.615240 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa5b9a-f21d-4f47-b063-e90bb0dfb658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:31.615893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa5b9a-f21d-4f47-b063-e90bb0dfb658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:31.616236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa5b9a-f21d-4f47-b063-e90bb0dfb658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:31.617391 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa5b9a-f21d-4f47-b063-e90bb0dfb658 admin admin] PATH_INFO: `/v3/users/2eb470b36e77498491d34abeef7e4314` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:31.618653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa5b9a-f21d-4f47-b063-e90bb0dfb658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eb470b36e77498491d34abeef7e4314)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:31.621350 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa5b9a-f21d-4f47-b063-e90bb0dfb658 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:31.625398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:31.723453 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bfaa5b9a-f21d-4f47-b063-e90bb0dfb658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eb470b36e77498491d34abeef7e4314'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:31.734413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bfaa5b9a-f21d-4f47-b063-e90bb0dfb658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eb470b36e77498491d34abeef7e4314'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:31.735380 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bfaa5b9a-f21d-4f47-b063-e90bb0dfb658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eb470b36e77498491d34abeef7e4314'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:31.832064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bfaa5b9a-f21d-4f47-b063-e90bb0dfb658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eb470b36e77498491d34abeef7e4314'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:31.832541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bfaa5b9a-f21d-4f47-b063-e90bb0dfb658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eb470b36e77498491d34abeef7e4314'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:31.861459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 894/1764] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:31 2022] DELETE /identity/v3/users/2eb470b36e77498491d34abeef7e4314 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:31.864247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffbeab-4d27-417b-a4f5-eb33d0b9f29a None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:31.864247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffbeab-4d27-417b-a4f5-eb33d0b9f29a None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:31.864828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffbeab-4d27-417b-a4f5-eb33d0b9f29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:31.923992 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-64ffbeab-4d27-417b-a4f5-eb33d0b9f29a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:31.925352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-64ffbeab-4d27-417b-a4f5-eb33d0b9f29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:32.083226 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 871/1765] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:32.095742 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa5b9a-f21d-4f47-b063-e90bb0dfb658 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:32.100261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-bfaa5b9a-f21d-4f47-b063-e90bb0dfb658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:32.248369 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa5b9a-f21d-4f47-b063-e90bb0dfb658 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:32.255891 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928e2a0-e291-46b2-8bc4-bdce240319b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:32.256569 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e2a0-e291-46b2-8bc4-bdce240319b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:32.256928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e2a0-e291-46b2-8bc4-bdce240319b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:32.257091 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e2a0-e291-46b2-8bc4-bdce240319b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:32.272492 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928e2a0-e291-46b2-8bc4-bdce240319b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:32.275121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928e2a0-e291-46b2-8bc4-bdce240319b6 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:32.301199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 895/1766] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:32.754265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffbeab-4d27-417b-a4f5-eb33d0b9f29a None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:32.755824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-64ffbeab-4d27-417b-a4f5-eb33d0b9f29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:32.769841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffbeab-4d27-417b-a4f5-eb33d0b9f29a None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:32.775470 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517e64b-66f7-41fa-a6af-789b251a52b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:32.776124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517e64b-66f7-41fa-a6af-789b251a52b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:32.776404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517e64b-66f7-41fa-a6af-789b251a52b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:32.776674 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517e64b-66f7-41fa-a6af-789b251a52b2 admin admin] PATH_INFO: `/v3/projects/78e15d310c6549579aa79a554475e041` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:32.777980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517e64b-66f7-41fa-a6af-789b251a52b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78e15d310c6549579aa79a554475e041)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:32.779822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517e64b-66f7-41fa-a6af-789b251a52b2 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:32.843123 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0517e64b-66f7-41fa-a6af-789b251a52b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78e15d310c6549579aa79a554475e041', 'request_id': 'req-0517e64b-66f7-41fa-a6af-789b251a52b2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:32.850699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0517e64b-66f7-41fa-a6af-789b251a52b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78e15d310c6549579aa79a554475e041', 'request_id': 'req-0517e64b-66f7-41fa-a6af-789b251a52b2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:32.858921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0517e64b-66f7-41fa-a6af-789b251a52b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78e15d310c6549579aa79a554475e041', 'request_id': 'req-0517e64b-66f7-41fa-a6af-789b251a52b2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:32.870632 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:32.877960 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0517e64b-66f7-41fa-a6af-789b251a52b2 admin admin] The token cache is being invalidate because project 78e15d310c6549579aa79a554475e041 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:32.879432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0517e64b-66f7-41fa-a6af-789b251a52b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:32.881296 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0517e64b-66f7-41fa-a6af-789b251a52b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:32.884446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 872/1767] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:32 2022] DELETE /identity/v3/projects/78e15d310c6549579aa79a554475e041 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:32.888529 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928e2a0-e291-46b2-8bc4-bdce240319b6 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:32.899251 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928e2a0-e291-46b2-8bc4-bdce240319b6 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:32.979928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57979bb-1f07-4e55-be0f-f846cdeeb786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:32.980600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57979bb-1f07-4e55-be0f-f846cdeeb786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:32.980882 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57979bb-1f07-4e55-be0f-f846cdeeb786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:32.981204 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57979bb-1f07-4e55-be0f-f846cdeeb786 admin admin] PATH_INFO: `/v3/users/c182d62b14ef4d9db495cc89844c1050` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:32.982526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57979bb-1f07-4e55-be0f-f846cdeeb786 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c182d62b14ef4d9db495cc89844c1050)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:32.984570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57979bb-1f07-4e55-be0f-f846cdeeb786 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:33.050590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:33.086969 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e57979bb-1f07-4e55-be0f-f846cdeeb786 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c182d62b14ef4d9db495cc89844c1050'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:33.101378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e57979bb-1f07-4e55-be0f-f846cdeeb786 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c182d62b14ef4d9db495cc89844c1050'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:33.102054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e57979bb-1f07-4e55-be0f-f846cdeeb786 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c182d62b14ef4d9db495cc89844c1050'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:33.130907 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e57979bb-1f07-4e55-be0f-f846cdeeb786 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c182d62b14ef4d9db495cc89844c1050'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:33.132253 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e57979bb-1f07-4e55-be0f-f846cdeeb786 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c182d62b14ef4d9db495cc89844c1050'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:33.158984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 896/1768] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:32 2022] DELETE /identity/v3/users/c182d62b14ef4d9db495cc89844c1050 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:33.588209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517e64b-66f7-41fa-a6af-789b251a52b2 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:33.589695 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0517e64b-66f7-41fa-a6af-789b251a52b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:33.607575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517e64b-66f7-41fa-a6af-789b251a52b2 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:33.674579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf15f3-3266-49db-b282-bdf6202f28b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:33.675631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf15f3-3266-49db-b282-bdf6202f28b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:33.676089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf15f3-3266-49db-b282-bdf6202f28b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:33.676537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf15f3-3266-49db-b282-bdf6202f28b9 admin admin] PATH_INFO: `/v3/projects/f92e28ecccf748fbb0dd0fb5761e642d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:33.678351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf15f3-3266-49db-b282-bdf6202f28b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f92e28ecccf748fbb0dd0fb5761e642d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:33.682432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf15f3-3266-49db-b282-bdf6202f28b9 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:33.728270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8cbf15f3-3266-49db-b282-bdf6202f28b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f92e28ecccf748fbb0dd0fb5761e642d', 'request_id': 'req-8cbf15f3-3266-49db-b282-bdf6202f28b9'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:33.734453 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8cbf15f3-3266-49db-b282-bdf6202f28b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f92e28ecccf748fbb0dd0fb5761e642d', 'request_id': 'req-8cbf15f3-3266-49db-b282-bdf6202f28b9'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:33.740286 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8cbf15f3-3266-49db-b282-bdf6202f28b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f92e28ecccf748fbb0dd0fb5761e642d', 'request_id': 'req-8cbf15f3-3266-49db-b282-bdf6202f28b9'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:33.756100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8cbf15f3-3266-49db-b282-bdf6202f28b9 admin admin] The token cache is being invalidate because project f92e28ecccf748fbb0dd0fb5761e642d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:33.756510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8cbf15f3-3266-49db-b282-bdf6202f28b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:33.757298 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8cbf15f3-3266-49db-b282-bdf6202f28b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:33.759458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 873/1769] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:33 2022] DELETE /identity/v3/projects/f92e28ecccf748fbb0dd0fb5761e642d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:33.764472 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724ae08-b809-46b1-a229-1ce2e2d1fb21 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:33.764756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724ae08-b809-46b1-a229-1ce2e2d1fb21 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:33.765034 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724ae08-b809-46b1-a229-1ce2e2d1fb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:33.790193 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-c724ae08-b809-46b1-a229-1ce2e2d1fb21 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:33.791742 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-c724ae08-b809-46b1-a229-1ce2e2d1fb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:33.881150 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 897/1770] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:33.888931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ccda5-22e7-4b81-8d07-4a15c6f8b7c7 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:33.889437 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ccda5-22e7-4b81-8d07-4a15c6f8b7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:33.889524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ccda5-22e7-4b81-8d07-4a15c6f8b7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:33.912683 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-526ccda5-22e7-4b81-8d07-4a15c6f8b7c7 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:33.915643 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-526ccda5-22e7-4b81-8d07-4a15c6f8b7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:33.931854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 874/1771] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:33.937902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724ae08-b809-46b1-a229-1ce2e2d1fb21 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:33.948875 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724ae08-b809-46b1-a229-1ce2e2d1fb21 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:33.965736 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd297e0a-d63a-4ebc-b354-ff1f045a1972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:33.966393 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd297e0a-d63a-4ebc-b354-ff1f045a1972 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:33.966660 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd297e0a-d63a-4ebc-b354-ff1f045a1972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:33.966996 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd297e0a-d63a-4ebc-b354-ff1f045a1972 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:33.967649 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd297e0a-d63a-4ebc-b354-ff1f045a1972 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:33.967952 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd297e0a-d63a-4ebc-b354-ff1f045a1972 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:33.969727 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd297e0a-d63a-4ebc-b354-ff1f045a1972 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:34.000065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 898/1772] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:34.000503 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ccda5-22e7-4b81-8d07-4a15c6f8b7c7 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:34.013189 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ccda5-22e7-4b81-8d07-4a15c6f8b7c7 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:34.021627 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c656dee-d6ec-43be-bb6a-1f8080d1eaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:34.023186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c656dee-d6ec-43be-bb6a-1f8080d1eaac admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:34.023709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c656dee-d6ec-43be-bb6a-1f8080d1eaac admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:34.024217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c656dee-d6ec-43be-bb6a-1f8080d1eaac admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:34.026366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c656dee-d6ec-43be-bb6a-1f8080d1eaac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:34.046523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c656dee-d6ec-43be-bb6a-1f8080d1eaac admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:34.138279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 875/1773] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:33 2022] POST /identity/v3/projects => generated 369 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:34.139850 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd297e0a-d63a-4ebc-b354-ff1f045a1972 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:34.153284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd297e0a-d63a-4ebc-b354-ff1f045a1972 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:34.357252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4bfd7-94e1-466e-84e4-6439b0ba8fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:34.357741 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4bfd7-94e1-466e-84e4-6439b0ba8fb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:34.358078 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4bfd7-94e1-466e-84e4-6439b0ba8fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:34.358360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4bfd7-94e1-466e-84e4-6439b0ba8fb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:34.359137 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba4bfd7-94e1-466e-84e4-6439b0ba8fb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:34.361404 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba4bfd7-94e1-466e-84e4-6439b0ba8fb8 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:34.403399 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 899/1774] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:34 2022] POST /identity/v3/users => generated 285 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:34.408595 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c656dee-d6ec-43be-bb6a-1f8080d1eaac admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:34.423007 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c656dee-d6ec-43be-bb6a-1f8080d1eaac admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:34.428499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdb39b-caba-4b52-9c71-32a8f9388659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:34.429392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdb39b-caba-4b52-9c71-32a8f9388659 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:34.430141 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdb39b-caba-4b52-9c71-32a8f9388659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:34.430913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdb39b-caba-4b52-9c71-32a8f9388659 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:34.431747 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbdb39b-caba-4b52-9c71-32a8f9388659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:34.433989 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbdb39b-caba-4b52-9c71-32a8f9388659 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:34.446395 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 876/1775] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:34.455353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4bfd7-94e1-466e-84e4-6439b0ba8fb8 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:34.469639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4bfd7-94e1-466e-84e4-6439b0ba8fb8 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:34.476701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f1a333-f57f-4924-b569-c3e2d4a04117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:34.478241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1a333-f57f-4924-b569-c3e2d4a04117 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:34.478806 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1a333-f57f-4924-b569-c3e2d4a04117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:34.479302 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f1a333-f57f-4924-b569-c3e2d4a04117 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:34.480171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1a333-f57f-4924-b569-c3e2d4a04117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:34.482013 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f1a333-f57f-4924-b569-c3e2d4a04117 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:34.499016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 900/1776] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:34.501488 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdb39b-caba-4b52-9c71-32a8f9388659 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:34.512425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdb39b-caba-4b52-9c71-32a8f9388659 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:34.517426 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83f24f-3b4c-4e4c-9366-0b7c21aaf474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:34.518100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83f24f-3b4c-4e4c-9366-0b7c21aaf474 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:34.518371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83f24f-3b4c-4e4c-9366-0b7c21aaf474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:34.518640 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83f24f-3b4c-4e4c-9366-0b7c21aaf474 admin admin] PATH_INFO: `/v3/projects/b4ec3afad39045ae9a8f969a9cbfe336/users/c7675300b892453abe001a8fad0a0808/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:34.538275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83f24f-3b4c-4e4c-9366-0b7c21aaf474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4ec3afad39045ae9a8f969a9cbfe336, user_id=c7675300b892453abe001a8fad0a0808, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:34.541258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83f24f-3b4c-4e4c-9366-0b7c21aaf474 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:34.571729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 877/1777] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:34 2022] PUT /identity/v3/projects/b4ec3afad39045ae9a8f969a9cbfe336/users/c7675300b892453abe001a8fad0a0808/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:34.582355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9a037-18c6-40fc-9023-39e2257a6e43 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:34.583179 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9a037-18c6-40fc-9023-39e2257a6e43 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:34.583706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9a037-18c6-40fc-9023-39e2257a6e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:34.635784 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:34.653932 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-89e9a037-18c6-40fc-9023-39e2257a6e43 None None] MFA Rules not processed for user `c7675300b892453abe001a8fad0a0808`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:34.655572 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-89e9a037-18c6-40fc-9023-39e2257a6e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:34.793329 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 901/1778] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:34.798637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83f24f-3b4c-4e4c-9366-0b7c21aaf474 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:34.905314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:34.948250 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9a037-18c6-40fc-9023-39e2257a6e43 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:34.953788 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-89e9a037-18c6-40fc-9023-39e2257a6e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:35.015674 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83f24f-3b4c-4e4c-9366-0b7c21aaf474 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:35.032354 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167987a-95c4-43f1-81d7-8d4e444ace73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:35.032915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167987a-95c4-43f1-81d7-8d4e444ace73 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:35.033254 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167987a-95c4-43f1-81d7-8d4e444ace73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:35.033703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167987a-95c4-43f1-81d7-8d4e444ace73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:35.034344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167987a-95c4-43f1-81d7-8d4e444ace73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:35.036907 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167987a-95c4-43f1-81d7-8d4e444ace73 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:35.075714 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9a037-18c6-40fc-9023-39e2257a6e43 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:35.094315 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 878/1779] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:34 2022] POST /identity/v3/projects => generated 367 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:35.097519 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698456e-b096-46b0-be5c-43e473ea39d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:35.102828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698456e-b096-46b0-be5c-43e473ea39d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:35.103032 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167987a-95c4-43f1-81d7-8d4e444ace73 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:35.103140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698456e-b096-46b0-be5c-43e473ea39d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:35.103140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698456e-b096-46b0-be5c-43e473ea39d4 admin admin] PATH_INFO: `/v3/users/debc8889dd0e4979a1d7cce8dafe48cc` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:35.104345 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698456e-b096-46b0-be5c-43e473ea39d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=debc8889dd0e4979a1d7cce8dafe48cc)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:35.106527 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698456e-b096-46b0-be5c-43e473ea39d4 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:35.209829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0698456e-b096-46b0-be5c-43e473ea39d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'debc8889dd0e4979a1d7cce8dafe48cc'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:35.210781 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167987a-95c4-43f1-81d7-8d4e444ace73 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:35.216471 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1b8d8-6f77-4709-bd8c-d16ef3be173d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:35.217073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1b8d8-6f77-4709-bd8c-d16ef3be173d admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:35.217350 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1b8d8-6f77-4709-bd8c-d16ef3be173d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:35.217683 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1b8d8-6f77-4709-bd8c-d16ef3be173d admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:35.218368 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1b8d8-6f77-4709-bd8c-d16ef3be173d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:35.220482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1b8d8-6f77-4709-bd8c-d16ef3be173d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:35.224613 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0698456e-b096-46b0-be5c-43e473ea39d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'debc8889dd0e4979a1d7cce8dafe48cc'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:35.226308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0698456e-b096-46b0-be5c-43e473ea39d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'debc8889dd0e4979a1d7cce8dafe48cc'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:35.251568 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0698456e-b096-46b0-be5c-43e473ea39d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'debc8889dd0e4979a1d7cce8dafe48cc'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:35.253875 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0698456e-b096-46b0-be5c-43e473ea39d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'debc8889dd0e4979a1d7cce8dafe48cc'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:35.261589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 879/1780] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:35 2022] POST /identity/v3/users => generated 290 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:35.263704 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1b8d8-6f77-4709-bd8c-d16ef3be173d admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:35.271634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-10e1b8d8-6f77-4709-bd8c-d16ef3be173d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:35.304615 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1b8d8-6f77-4709-bd8c-d16ef3be173d admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:35.305761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 902/1781] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:34 2022] DELETE /identity/v3/users/debc8889dd0e4979a1d7cce8dafe48cc => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:35.312026 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1393a6a-58d8-4f6b-ad87-801a4daaae72 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:35.312441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1393a6a-58d8-4f6b-ad87-801a4daaae72 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:35.312823 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1393a6a-58d8-4f6b-ad87-801a4daaae72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:35.369017 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-f1393a6a-58d8-4f6b-ad87-801a4daaae72 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:35.370335 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-f1393a6a-58d8-4f6b-ad87-801a4daaae72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:35.391515 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:35.450681 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43730f6-c716-419e-ad9c-e2c8d1f20b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:35.451822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43730f6-c716-419e-ad9c-e2c8d1f20b06 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:35.452110 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43730f6-c716-419e-ad9c-e2c8d1f20b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:35.452386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43730f6-c716-419e-ad9c-e2c8d1f20b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:35.453289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43730f6-c716-419e-ad9c-e2c8d1f20b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:35.455804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43730f6-c716-419e-ad9c-e2c8d1f20b06 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:35.482717 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 880/1782] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:35 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:35.491646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43730f6-c716-419e-ad9c-e2c8d1f20b06 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:35.497900 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 903/1783] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:35.511146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1393a6a-58d8-4f6b-ad87-801a4daaae72 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:35.512707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-f1393a6a-58d8-4f6b-ad87-801a4daaae72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:35.531064 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43730f6-c716-419e-ad9c-e2c8d1f20b06 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:35.547250 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17da5e-5f0c-4613-9716-d24290760427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:35.548277 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17da5e-5f0c-4613-9716-d24290760427 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:35.548890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17da5e-5f0c-4613-9716-d24290760427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:35.549375 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17da5e-5f0c-4613-9716-d24290760427 admin admin] PATH_INFO: `/v3/projects/f2ff22a8aa6e4b7cb3be2ded8a76daa7/users/ccd80e067214498396d7c1a0e5ce00bd/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:35.591920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17da5e-5f0c-4613-9716-d24290760427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2ff22a8aa6e4b7cb3be2ded8a76daa7, user_id=ccd80e067214498396d7c1a0e5ce00bd, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:35.596630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17da5e-5f0c-4613-9716-d24290760427 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:35.630141 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1393a6a-58d8-4f6b-ad87-801a4daaae72 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:35.645676 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 881/1784] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:35 2022] PUT /identity/v3/projects/f2ff22a8aa6e4b7cb3be2ded8a76daa7/users/ccd80e067214498396d7c1a0e5ce00bd/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:35.650097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17da5e-5f0c-4613-9716-d24290760427 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:35.699183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:35.711339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b8aa1-4a89-4dc5-b9bd-610c140b6e0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:35.711964 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b8aa1-4a89-4dc5-b9bd-610c140b6e0d service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:35.712244 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b8aa1-4a89-4dc5-b9bd-610c140b6e0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:35.712516 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b8aa1-4a89-4dc5-b9bd-610c140b6e0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:35.841412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17da5e-5f0c-4613-9716-d24290760427 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:35.856292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b01f1a-ea07-4bf6-b859-ed6eca764e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:35.856924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b01f1a-ea07-4bf6-b859-ed6eca764e77 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:35.858071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b01f1a-ea07-4bf6-b859-ed6eca764e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:35.858071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b01f1a-ea07-4bf6-b859-ed6eca764e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:35.858422 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b01f1a-ea07-4bf6-b859-ed6eca764e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:35.860233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b01f1a-ea07-4bf6-b859-ed6eca764e77 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:35.867722 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b8aa1-4a89-4dc5-b9bd-610c140b6e0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:35.869722 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b8aa1-4a89-4dc5-b9bd-610c140b6e0d service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:35.882512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 882/1785] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:35 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:35.884904 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 904/1786] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:35.889217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b01f1a-ea07-4bf6-b859-ed6eca764e77 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:35.910857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b01f1a-ea07-4bf6-b859-ed6eca764e77 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:35.922526 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649f134-f39d-4212-908f-4c76db6c2fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:35.923259 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649f134-f39d-4212-908f-4c76db6c2fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:35.923540 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649f134-f39d-4212-908f-4c76db6c2fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:35.923822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649f134-f39d-4212-908f-4c76db6c2fc0 admin admin] PATH_INFO: `/v3/projects/f2ff22a8aa6e4b7cb3be2ded8a76daa7/users/ccd80e067214498396d7c1a0e5ce00bd/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:35.927585 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649f134-f39d-4212-908f-4c76db6c2fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2ff22a8aa6e4b7cb3be2ded8a76daa7, user_id=ccd80e067214498396d7c1a0e5ce00bd, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:35.938208 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649f134-f39d-4212-908f-4c76db6c2fc0 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:36.037290 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 883/1787] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:35 2022] PUT /identity/v3/projects/f2ff22a8aa6e4b7cb3be2ded8a76daa7/users/ccd80e067214498396d7c1a0e5ce00bd/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:36.049708 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0b7ec-da5b-4ce1-9815-0e1d0eecf957 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:36.050302 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0b7ec-da5b-4ce1-9815-0e1d0eecf957 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:36.050302 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0b7ec-da5b-4ce1-9815-0e1d0eecf957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:36.085812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-55e0b7ec-da5b-4ce1-9815-0e1d0eecf957 None None] MFA Rules not processed for user `ccd80e067214498396d7c1a0e5ce00bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:36.087330 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-55e0b7ec-da5b-4ce1-9815-0e1d0eecf957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:36.108631 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:36.198869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 905/1788] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:36.205497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649f134-f39d-4212-908f-4c76db6c2fc0 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:36.208654 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d649f134-f39d-4212-908f-4c76db6c2fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:36.287733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649f134-f39d-4212-908f-4c76db6c2fc0 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:36.293296 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac91eee-8f37-41dc-a4c0-de158446700d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:36.293944 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac91eee-8f37-41dc-a4c0-de158446700d service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:36.294225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac91eee-8f37-41dc-a4c0-de158446700d service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:36.294497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac91eee-8f37-41dc-a4c0-de158446700d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:36.311193 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac91eee-8f37-41dc-a4c0-de158446700d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:36.312713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac91eee-8f37-41dc-a4c0-de158446700d service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:36.332443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 884/1789] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:36.577980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0b7ec-da5b-4ce1-9815-0e1d0eecf957 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:36.579363 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-55e0b7ec-da5b-4ce1-9815-0e1d0eecf957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:36.716537 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0b7ec-da5b-4ce1-9815-0e1d0eecf957 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:36.724894 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b30ee-9c44-4065-8af8-79c4d4b94d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:36.726373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b30ee-9c44-4065-8af8-79c4d4b94d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:36.726725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b30ee-9c44-4065-8af8-79c4d4b94d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:36.727603 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b30ee-9c44-4065-8af8-79c4d4b94d33 admin admin] PATH_INFO: `/v3/projects/9a99dffe9d0c406fb4fbd51a8406f7f2` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:36.729208 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b30ee-9c44-4065-8af8-79c4d4b94d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a99dffe9d0c406fb4fbd51a8406f7f2)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:36.731459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b30ee-9c44-4065-8af8-79c4d4b94d33 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:36.877637 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-508b30ee-9c44-4065-8af8-79c4d4b94d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a99dffe9d0c406fb4fbd51a8406f7f2', 'request_id': 'req-508b30ee-9c44-4065-8af8-79c4d4b94d33'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:36.891087 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-508b30ee-9c44-4065-8af8-79c4d4b94d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a99dffe9d0c406fb4fbd51a8406f7f2', 'request_id': 'req-508b30ee-9c44-4065-8af8-79c4d4b94d33'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:36.902780 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-508b30ee-9c44-4065-8af8-79c4d4b94d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a99dffe9d0c406fb4fbd51a8406f7f2', 'request_id': 'req-508b30ee-9c44-4065-8af8-79c4d4b94d33'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:36.958562 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-508b30ee-9c44-4065-8af8-79c4d4b94d33 admin admin] The token cache is being invalidate because project 9a99dffe9d0c406fb4fbd51a8406f7f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:36.959267 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-508b30ee-9c44-4065-8af8-79c4d4b94d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:36.961159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-508b30ee-9c44-4065-8af8-79c4d4b94d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:36.966202 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 906/1790] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:36 2022] DELETE /identity/v3/projects/9a99dffe9d0c406fb4fbd51a8406f7f2 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:36.971206 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac91eee-8f37-41dc-a4c0-de158446700d service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:36.992834 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac91eee-8f37-41dc-a4c0-de158446700d service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:37.142067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078b631-b000-4d2b-a972-2a9137780a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:37.143036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078b631-b000-4d2b-a972-2a9137780a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:37.143699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078b631-b000-4d2b-a972-2a9137780a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:37.144515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078b631-b000-4d2b-a972-2a9137780a87 admin admin] PATH_INFO: `/v3/users/09f3966f30494eacbad130db240b86eb` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:37.148766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078b631-b000-4d2b-a972-2a9137780a87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09f3966f30494eacbad130db240b86eb)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:37.152875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078b631-b000-4d2b-a972-2a9137780a87 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:37.259702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:37.285684 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c078b631-b000-4d2b-a972-2a9137780a87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09f3966f30494eacbad130db240b86eb'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:37.311836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c078b631-b000-4d2b-a972-2a9137780a87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09f3966f30494eacbad130db240b86eb'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:37.323517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c078b631-b000-4d2b-a972-2a9137780a87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09f3966f30494eacbad130db240b86eb'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:37.356989 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c078b631-b000-4d2b-a972-2a9137780a87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09f3966f30494eacbad130db240b86eb'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:37.357707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c078b631-b000-4d2b-a972-2a9137780a87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09f3966f30494eacbad130db240b86eb'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:37.382887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 885/1791] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:36 2022] DELETE /identity/v3/users/09f3966f30494eacbad130db240b86eb => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:37.606068 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba27a3-e91c-4320-8ce3-3771be8fae9a None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:37.606552 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba27a3-e91c-4320-8ce3-3771be8fae9a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:37.607682 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba27a3-e91c-4320-8ce3-3771be8fae9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:37.696415 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-a8ba27a3-e91c-4320-8ce3-3771be8fae9a None None] MFA Rules not processed for user `f77bbd8de87042aab3674db366635cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:37.698640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a8ba27a3-e91c-4320-8ce3-3771be8fae9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:37.772243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:37.857639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 907/1792] 10.222.0.67 () {58 vars in 1015 bytes} [Thu Sep 22 14:58:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:37.911165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078b631-b000-4d2b-a972-2a9137780a87 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:37.911165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c078b631-b000-4d2b-a972-2a9137780a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:37.929508 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba27a3-e91c-4320-8ce3-3771be8fae9a None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:37.931583 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078b631-b000-4d2b-a972-2a9137780a87 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:37.936631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ed34c-6ef7-43d9-81a7-e0af71c67fe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:37.937388 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ed34c-6ef7-43d9-81a7-e0af71c67fe1 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:37.937658 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ed34c-6ef7-43d9-81a7-e0af71c67fe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:37.938018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ed34c-6ef7-43d9-81a7-e0af71c67fe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:37.939540 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba27a3-e91c-4320-8ce3-3771be8fae9a None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:37.949772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ed34c-6ef7-43d9-81a7-e0af71c67fe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:37.951616 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ed34c-6ef7-43d9-81a7-e0af71c67fe1 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:37.959335 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8b3e1-8482-4e8b-ab6f-9374d7fb654e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:37.959335 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8b3e1-8482-4e8b-ab6f-9374d7fb654e service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:37.960172 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8b3e1-8482-4e8b-ab6f-9374d7fb654e service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:37.960458 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8b3e1-8482-4e8b-ab6f-9374d7fb654e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:37.963093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 886/1793] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:37.964124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ed34c-6ef7-43d9-81a7-e0af71c67fe1 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:37.965833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-682ed34c-6ef7-43d9-81a7-e0af71c67fe1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:37.980729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ed34c-6ef7-43d9-81a7-e0af71c67fe1 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:37.985047 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c2cba-fe5a-4912-81e1-ecf37dabb46d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:37.985727 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c2cba-fe5a-4912-81e1-ecf37dabb46d service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:37.986008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c2cba-fe5a-4912-81e1-ecf37dabb46d service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:37.986280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c2cba-fe5a-4912-81e1-ecf37dabb46d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:37.998337 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8b3e1-8482-4e8b-ab6f-9374d7fb654e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:37.999196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c2cba-fe5a-4912-81e1-ecf37dabb46d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:37.999933 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8b3e1-8482-4e8b-ab6f-9374d7fb654e service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:38.001922 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c2cba-fe5a-4912-81e1-ecf37dabb46d service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:38.022559 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 887/1794] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:38.045532 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 908/1795] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:39.071785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c2cba-fe5a-4912-81e1-ecf37dabb46d service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:39.076929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-793c2cba-fe5a-4912-81e1-ecf37dabb46d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:39.312446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c2cba-fe5a-4912-81e1-ecf37dabb46d service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:39.333930 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad030a9b-d979-4076-93d9-00166e37b19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:39.335477 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad030a9b-d979-4076-93d9-00166e37b19c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:39.336886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad030a9b-d979-4076-93d9-00166e37b19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:39.337390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad030a9b-d979-4076-93d9-00166e37b19c admin admin] PATH_INFO: `/v3/users/c7675300b892453abe001a8fad0a0808` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:39.338877 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad030a9b-d979-4076-93d9-00166e37b19c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7675300b892453abe001a8fad0a0808)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:39.341980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad030a9b-d979-4076-93d9-00166e37b19c admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:39.500458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ad030a9b-d979-4076-93d9-00166e37b19c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7675300b892453abe001a8fad0a0808'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:39.523478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ad030a9b-d979-4076-93d9-00166e37b19c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7675300b892453abe001a8fad0a0808'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:39.545714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ad030a9b-d979-4076-93d9-00166e37b19c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7675300b892453abe001a8fad0a0808'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:39.602843 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ad030a9b-d979-4076-93d9-00166e37b19c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7675300b892453abe001a8fad0a0808'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:39.604578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ad030a9b-d979-4076-93d9-00166e37b19c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7675300b892453abe001a8fad0a0808'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:39.629675 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8b3e1-8482-4e8b-ab6f-9374d7fb654e service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:39.635947 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-42b8b3e1-8482-4e8b-ab6f-9374d7fb654e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:39.641785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 888/1796] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:39 2022] DELETE /identity/v3/users/c7675300b892453abe001a8fad0a0808 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:39.647369 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4d058-7792-4122-9b4b-8ae14a41e32d None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:39.647634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4d058-7792-4122-9b4b-8ae14a41e32d None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:39.647880 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4d058-7792-4122-9b4b-8ae14a41e32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:39.668753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8b3e1-8482-4e8b-ab6f-9374d7fb654e service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:39.743246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-95e4d058-7792-4122-9b4b-8ae14a41e32d None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:39.745795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-95e4d058-7792-4122-9b4b-8ae14a41e32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:39.752119 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:39.846766 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:39.847447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:39.847737 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:39.848039 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0 admin admin] PATH_INFO: `/v3/projects/3da5c029dcf64920aa67c56b1b007806` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:39.855963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3da5c029dcf64920aa67c56b1b007806)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:39.858763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:39.892655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 889/1797] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:39.981855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3da5c029dcf64920aa67c56b1b007806', 'request_id': 'req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.002057 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4d058-7792-4122-9b4b-8ae14a41e32d None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:40.003744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-95e4d058-7792-4122-9b4b-8ae14a41e32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:40.008480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3da5c029dcf64920aa67c56b1b007806', 'request_id': 'req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.028203 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3da5c029dcf64920aa67c56b1b007806', 'request_id': 'req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.139697 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4d058-7792-4122-9b4b-8ae14a41e32d None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:40.141988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0 admin admin] The token cache is being invalidate because project 3da5c029dcf64920aa67c56b1b007806 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:40.144458 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.148690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.155909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 909/1798] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:39 2022] DELETE /identity/v3/projects/3da5c029dcf64920aa67c56b1b007806 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:40.159427 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:40.166388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:40.300244 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56144383-8a5a-4a5b-b335-ad0262abcad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:40.300875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56144383-8a5a-4a5b-b335-ad0262abcad6 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:40.305545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56144383-8a5a-4a5b-b335-ad0262abcad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:40.305545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56144383-8a5a-4a5b-b335-ad0262abcad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:40.307303 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-56144383-8a5a-4a5b-b335-ad0262abcad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:40.438272 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6d7cc-80c6-49b4-a3e3-716676ef65e0 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:40.444931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56144383-8a5a-4a5b-b335-ad0262abcad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:40.448979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56144383-8a5a-4a5b-b335-ad0262abcad6 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:40.453361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb218bcf-de8a-4f19-8f7a-efc5a59d6f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:40.456107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb218bcf-de8a-4f19-8f7a-efc5a59d6f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:40.456646 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb218bcf-de8a-4f19-8f7a-efc5a59d6f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:40.458503 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb218bcf-de8a-4f19-8f7a-efc5a59d6f30 admin admin] PATH_INFO: `/v3/users/92d3c59f36be4b46859bb29397fd728c` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:40.463437 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb218bcf-de8a-4f19-8f7a-efc5a59d6f30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92d3c59f36be4b46859bb29397fd728c)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:40.471531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb218bcf-de8a-4f19-8f7a-efc5a59d6f30 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:40.484498 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 890/1799] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:40.488928 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596dbdc-58dc-48d3-9a1e-c4d551272eae None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:40.489932 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596dbdc-58dc-48d3-9a1e-c4d551272eae None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:40.490859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596dbdc-58dc-48d3-9a1e-c4d551272eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:40.494102 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:40.548421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-a596dbdc-58dc-48d3-9a1e-c4d551272eae None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:40.549855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a596dbdc-58dc-48d3-9a1e-c4d551272eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:40.561594 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 891/1800] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:40.571291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e9512-2e3b-47f5-b0ce-320d5ddbe202 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:40.571753 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e9512-2e3b-47f5-b0ce-320d5ddbe202 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:40.572265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e9512-2e3b-47f5-b0ce-320d5ddbe202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:40.602402 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-196e9512-2e3b-47f5-b0ce-320d5ddbe202 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:40.606498 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-196e9512-2e3b-47f5-b0ce-320d5ddbe202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:40.618701 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 892/1801] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:40.622879 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e9512-2e3b-47f5-b0ce-320d5ddbe202 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:40.629201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:40.630398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-fb218bcf-de8a-4f19-8f7a-efc5a59d6f30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92d3c59f36be4b46859bb29397fd728c'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.653055 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-fb218bcf-de8a-4f19-8f7a-efc5a59d6f30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92d3c59f36be4b46859bb29397fd728c'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.654791 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-fb218bcf-de8a-4f19-8f7a-efc5a59d6f30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92d3c59f36be4b46859bb29397fd728c'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.681286 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e9512-2e3b-47f5-b0ce-320d5ddbe202 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:40.691775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c84d32-87bb-4bce-a680-cd82ccb6c170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:40.692738 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c84d32-87bb-4bce-a680-cd82ccb6c170 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:40.693175 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c84d32-87bb-4bce-a680-cd82ccb6c170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:40.693584 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c84d32-87bb-4bce-a680-cd82ccb6c170 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:40.695005 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c84d32-87bb-4bce-a680-cd82ccb6c170 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:40.695958 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c84d32-87bb-4bce-a680-cd82ccb6c170 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:40.701575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c84d32-87bb-4bce-a680-cd82ccb6c170 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:40.740947 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 893/1802] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:40.744639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-fb218bcf-de8a-4f19-8f7a-efc5a59d6f30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92d3c59f36be4b46859bb29397fd728c'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.745767 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c84d32-87bb-4bce-a680-cd82ccb6c170 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:40.748895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-fb218bcf-de8a-4f19-8f7a-efc5a59d6f30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92d3c59f36be4b46859bb29397fd728c'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:40.763260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-26c84d32-87bb-4bce-a680-cd82ccb6c170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:40.824118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c84d32-87bb-4bce-a680-cd82ccb6c170 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:40.832303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 910/1803] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:40 2022] DELETE /identity/v3/users/92d3c59f36be4b46859bb29397fd728c => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:40.838531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025f372-1dc7-4b7c-8662-5067601bda78 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:40.843045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025f372-1dc7-4b7c-8662-5067601bda78 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:40.843352 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025f372-1dc7-4b7c-8662-5067601bda78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:40.937353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-6025f372-1dc7-4b7c-8662-5067601bda78 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:40.944431 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-6025f372-1dc7-4b7c-8662-5067601bda78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:41.042654 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c209a6-5545-4988-bff4-1269a01ac5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:41.043500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c209a6-5545-4988-bff4-1269a01ac5d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.043775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c209a6-5545-4988-bff4-1269a01ac5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.044081 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c209a6-5545-4988-bff4-1269a01ac5d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.049014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c209a6-5545-4988-bff4-1269a01ac5d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:41.053849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c209a6-5545-4988-bff4-1269a01ac5d5 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:41.151058 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 894/1804] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:40 2022] POST /identity/v3/projects => generated 399 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:41.174077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c209a6-5545-4988-bff4-1269a01ac5d5 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:41.253623 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 911/1805] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:41.338833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c209a6-5545-4988-bff4-1269a01ac5d5 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:41.352065 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af131008-2370-46a9-b561-9482df56e20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:41.352566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af131008-2370-46a9-b561-9482df56e20a admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.352845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af131008-2370-46a9-b561-9482df56e20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.353159 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af131008-2370-46a9-b561-9482df56e20a admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.353773 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af131008-2370-46a9-b561-9482df56e20a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:41.355741 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af131008-2370-46a9-b561-9482df56e20a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:41.374306 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025f372-1dc7-4b7c-8662-5067601bda78 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:41.377642 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-6025f372-1dc7-4b7c-8662-5067601bda78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:41.422363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 895/1806] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:41 2022] POST /identity/v3/users => generated 300 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:41.427717 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af131008-2370-46a9-b561-9482df56e20a admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:41.467837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af131008-2370-46a9-b561-9482df56e20a admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:41.483809 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566988d1-3ad4-4710-ac0c-54467274330c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:41.484450 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566988d1-3ad4-4710-ac0c-54467274330c admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.484733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566988d1-3ad4-4710-ac0c-54467274330c admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.485006 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566988d1-3ad4-4710-ac0c-54467274330c admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.488638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566988d1-3ad4-4710-ac0c-54467274330c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:41.490340 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566988d1-3ad4-4710-ac0c-54467274330c admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:41.507477 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 896/1807] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:41.511806 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566988d1-3ad4-4710-ac0c-54467274330c admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:41.540756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566988d1-3ad4-4710-ac0c-54467274330c admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:41.556160 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40718f2a-9309-4906-b8a3-db86c58d68b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:41.556787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40718f2a-9309-4906-b8a3-db86c58d68b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.557062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40718f2a-9309-4906-b8a3-db86c58d68b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.557332 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40718f2a-9309-4906-b8a3-db86c58d68b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.558818 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40718f2a-9309-4906-b8a3-db86c58d68b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:41.560958 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40718f2a-9309-4906-b8a3-db86c58d68b7 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:41.590539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025f372-1dc7-4b7c-8662-5067601bda78 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:41.599077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 897/1808] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:41 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:41.599077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d676d-2e9f-4a6e-b640-c210403d70f9 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.599077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d676d-2e9f-4a6e-b640-c210403d70f9 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.599077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d676d-2e9f-4a6e-b640-c210403d70f9 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.604570 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 898/1809] 10.222.0.26 () {54 vars in 921 bytes} [Thu Sep 22 14:58:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:41.606126 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d676d-2e9f-4a6e-b640-c210403d70f9 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:41.634600 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d676d-2e9f-4a6e-b640-c210403d70f9 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:41.651422 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3081f79a-cf2d-4dd9-ab40-43cc6968e8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:41.652109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081f79a-cf2d-4dd9-ab40-43cc6968e8ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.652385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081f79a-cf2d-4dd9-ab40-43cc6968e8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.652661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081f79a-cf2d-4dd9-ab40-43cc6968e8ba admin admin] PATH_INFO: `/v3/projects/57438b15417a4566bac79b68ce36c670/users/66b1c53f7fb140a8a4cd7ec7c26bc899/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.679412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:41.691053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3081f79a-cf2d-4dd9-ab40-43cc6968e8ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=57438b15417a4566bac79b68ce36c670, user_id=66b1c53f7fb140a8a4cd7ec7c26bc899, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:41.694276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3081f79a-cf2d-4dd9-ab40-43cc6968e8ba admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:41.697261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5aa588-ce33-435b-9196-975608374f21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:41.698088 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5aa588-ce33-435b-9196-975608374f21 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.698433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5aa588-ce33-435b-9196-975608374f21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.699654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5aa588-ce33-435b-9196-975608374f21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.730981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 899/1810] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:41 2022] PUT /identity/v3/projects/57438b15417a4566bac79b68ce36c670/users/66b1c53f7fb140a8a4cd7ec7c26bc899/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:41.732558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5aa588-ce33-435b-9196-975608374f21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:41.734288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5aa588-ce33-435b-9196-975608374f21 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:41.742397 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8d44c-a1a6-4541-8683-7f48376fe44a None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.742828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8d44c-a1a6-4541-8683-7f48376fe44a None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.743104 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8d44c-a1a6-4541-8683-7f48376fe44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.769980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 912/1811] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:41.775940 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c3c7f-6020-4fe0-954b-3e7588114e54 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:41.777073 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c3c7f-6020-4fe0-954b-3e7588114e54 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:41.777564 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c3c7f-6020-4fe0-954b-3e7588114e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:41.819491 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-ce1c3c7f-6020-4fe0-954b-3e7588114e54 None None] MFA Rules not processed for user `0c9be41c39494e54a14398a1f03c4d69`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:41.821271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ce1c3c7f-6020-4fe0-954b-3e7588114e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:41.832551 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-02d8d44c-a1a6-4541-8683-7f48376fe44a None None] MFA Rules not processed for user `66b1c53f7fb140a8a4cd7ec7c26bc899`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:41.833805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-02d8d44c-a1a6-4541-8683-7f48376fe44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:41.843552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:42.066966 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 900/1812] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:41 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:42.073221 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8d44c-a1a6-4541-8683-7f48376fe44a None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:42.073815 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 913/1813] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Sep 22 14:58:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:42.314881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c3c7f-6020-4fe0-954b-3e7588114e54 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:42.316638 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ce1c3c7f-6020-4fe0-954b-3e7588114e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:42.322436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8d44c-a1a6-4541-8683-7f48376fe44a None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:42.332486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192f34c-4905-41c3-8e98-777a6e149fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:42.333288 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192f34c-4905-41c3-8e98-777a6e149fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:42.333740 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192f34c-4905-41c3-8e98-777a6e149fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:42.334191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f192f34c-4905-41c3-8e98-777a6e149fe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:42.335217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192f34c-4905-41c3-8e98-777a6e149fe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:42.337309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f192f34c-4905-41c3-8e98-777a6e149fe8 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:42.356586 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c3c7f-6020-4fe0-954b-3e7588114e54 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:42.375083 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497ff6b-145c-434d-a8d9-2b88306e0646 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:42.378116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497ff6b-145c-434d-a8d9-2b88306e0646 service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:42.378503 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497ff6b-145c-434d-a8d9-2b88306e0646 service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:42.382808 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497ff6b-145c-434d-a8d9-2b88306e0646 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:42.423501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497ff6b-145c-434d-a8d9-2b88306e0646 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:42.425115 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497ff6b-145c-434d-a8d9-2b88306e0646 service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:42.446124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 901/1814] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:42 2022] POST /identity/v3/projects => generated 399 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:42.450555 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192f34c-4905-41c3-8e98-777a6e149fe8 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:42.464166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 914/1815] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:42.640978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f192f34c-4905-41c3-8e98-777a6e149fe8 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:42.655470 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063081ca-fbf3-403d-b072-5d221f7e47b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:42.656264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063081ca-fbf3-403d-b072-5d221f7e47b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:42.656558 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063081ca-fbf3-403d-b072-5d221f7e47b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:42.656972 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063081ca-fbf3-403d-b072-5d221f7e47b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:42.657895 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063081ca-fbf3-403d-b072-5d221f7e47b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:42.660222 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063081ca-fbf3-403d-b072-5d221f7e47b4 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:42.711409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497ff6b-145c-434d-a8d9-2b88306e0646 service nova] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:42.716538 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 902/1816] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:42 2022] POST /identity/v3/users => generated 306 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:42.744329 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497ff6b-145c-434d-a8d9-2b88306e0646 service nova] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:42.759647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe254ab-19ad-41af-9e6e-a0c1870d0839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:42.761778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe254ab-19ad-41af-9e6e-a0c1870d0839 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:42.762139 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe254ab-19ad-41af-9e6e-a0c1870d0839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:42.762661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe254ab-19ad-41af-9e6e-a0c1870d0839 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:42.763447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe254ab-19ad-41af-9e6e-a0c1870d0839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:42.768851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe254ab-19ad-41af-9e6e-a0c1870d0839 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:42.795762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 915/1817] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:42.799724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063081ca-fbf3-403d-b072-5d221f7e47b4 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:42.812714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063081ca-fbf3-403d-b072-5d221f7e47b4 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:42.827283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba6451-d5df-4989-9df1-8741e34d764a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:42.828242 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba6451-d5df-4989-9df1-8741e34d764a admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:42.828242 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba6451-d5df-4989-9df1-8741e34d764a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:42.830156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba6451-d5df-4989-9df1-8741e34d764a admin admin] PATH_INFO: `/v3/projects/234344d52dfd4d578fa9afef009c3713/users/77ed8dbff7824bd0b991816ada191a52/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:42.866952 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba6451-d5df-4989-9df1-8741e34d764a admin admin] RBAC: Authorizing `identity:create_grant(project_id=234344d52dfd4d578fa9afef009c3713, user_id=77ed8dbff7824bd0b991816ada191a52, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:42.874890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba6451-d5df-4989-9df1-8741e34d764a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:42.894917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:42.903688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 903/1818] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:42 2022] PUT /identity/v3/projects/234344d52dfd4d578fa9afef009c3713/users/77ed8dbff7824bd0b991816ada191a52/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:42.907988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe254ab-19ad-41af-9e6e-a0c1870d0839 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:42.928443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe254ab-19ad-41af-9e6e-a0c1870d0839 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:43.089808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010dcf8-9054-4932-a3e6-e63118351210 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:43.090346 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010dcf8-9054-4932-a3e6-e63118351210 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:43.090626 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010dcf8-9054-4932-a3e6-e63118351210 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:43.092880 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 904/1819] 10.222.0.26 () {54 vars in 921 bytes} [Thu Sep 22 14:58:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:43.117278 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d3245-a7c1-4942-b2ac-0ba8a088a5bb None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:43.117582 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d3245-a7c1-4942-b2ac-0ba8a088a5bb None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:43.118077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d3245-a7c1-4942-b2ac-0ba8a088a5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:43.155622 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-e40d3245-a7c1-4942-b2ac-0ba8a088a5bb None None] MFA Rules not processed for user `0c9be41c39494e54a14398a1f03c4d69`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:43.157806 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e40d3245-a7c1-4942-b2ac-0ba8a088a5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:43.254148 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130fd99-9a96-4545-9163-6209bcd41dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:43.254816 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130fd99-9a96-4545-9163-6209bcd41dda admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:43.256815 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130fd99-9a96-4545-9163-6209bcd41dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:43.258967 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130fd99-9a96-4545-9163-6209bcd41dda admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:43.260015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130fd99-9a96-4545-9163-6209bcd41dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:43.265012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130fd99-9a96-4545-9163-6209bcd41dda admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:43.281105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 916/1820] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:43.289357 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130fd99-9a96-4545-9163-6209bcd41dda admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:43.367726 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130fd99-9a96-4545-9163-6209bcd41dda admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:43.371807 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 905/1821] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Sep 22 14:58:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:43.388201 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b7d7e-0440-4efa-9b05-c72e1be60e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:43.389278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b7d7e-0440-4efa-9b05-c72e1be60e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:43.389684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b7d7e-0440-4efa-9b05-c72e1be60e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:43.390360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b7d7e-0440-4efa-9b05-c72e1be60e64 admin admin] PATH_INFO: `/v3/projects/234344d52dfd4d578fa9afef009c3713/users/77ed8dbff7824bd0b991816ada191a52/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:43.402567 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b7d7e-0440-4efa-9b05-c72e1be60e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=234344d52dfd4d578fa9afef009c3713, user_id=77ed8dbff7824bd0b991816ada191a52, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:43.407453 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b7d7e-0440-4efa-9b05-c72e1be60e64 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:43.452621 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 917/1822] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:43 2022] PUT /identity/v3/projects/234344d52dfd4d578fa9afef009c3713/users/77ed8dbff7824bd0b991816ada191a52/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:43.455668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c40e1-c545-4be9-a30e-fb80c3dbcd95 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:43.456118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c40e1-c545-4be9-a30e-fb80c3dbcd95 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:43.457801 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c40e1-c545-4be9-a30e-fb80c3dbcd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:43.528708 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-0c5c40e1-c545-4be9-a30e-fb80c3dbcd95 None None] MFA Rules not processed for user `77ed8dbff7824bd0b991816ada191a52`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:43.530187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0c5c40e1-c545-4be9-a30e-fb80c3dbcd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:43.665321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b7d7e-0440-4efa-9b05-c72e1be60e64 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:43.669529 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-027b7d7e-0440-4efa-9b05-c72e1be60e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:43.808220 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 906/1823] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:43 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:43.809285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c40e1-c545-4be9-a30e-fb80c3dbcd95 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:43.841891 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:43.901034 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b7d7e-0440-4efa-9b05-c72e1be60e64 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:43.915567 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:43.916240 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:43.916603 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:43.916943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa admin admin] PATH_INFO: `/v3/projects/b4ec3afad39045ae9a8f969a9cbfe336` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:43.918236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4ec3afad39045ae9a8f969a9cbfe336)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:43.920267 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:43.975686 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c40e1-c545-4be9-a30e-fb80c3dbcd95 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:43.987520 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def32ce-113d-4eb2-a2b7-1ae2adea9df8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:43.988501 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def32ce-113d-4eb2-a2b7-1ae2adea9df8 service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:43.989554 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def32ce-113d-4eb2-a2b7-1ae2adea9df8 service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:43.991178 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def32ce-113d-4eb2-a2b7-1ae2adea9df8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:44.005831 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def32ce-113d-4eb2-a2b7-1ae2adea9df8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:44.008973 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4ec3afad39045ae9a8f969a9cbfe336', 'request_id': 'req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.011878 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def32ce-113d-4eb2-a2b7-1ae2adea9df8 service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:44.021004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:44.024216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4ec3afad39045ae9a8f969a9cbfe336', 'request_id': 'req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.043649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 907/1824] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:44.046857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def32ce-113d-4eb2-a2b7-1ae2adea9df8 service nova] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:44.054012 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-8def32ce-113d-4eb2-a2b7-1ae2adea9df8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:44.057742 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4ec3afad39045ae9a8f969a9cbfe336', 'request_id': 'req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.120524 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa admin admin] The token cache is being invalidate because project b4ec3afad39045ae9a8f969a9cbfe336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:44.121182 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.123139 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.126985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 918/1825] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:43 2022] DELETE /identity/v3/projects/b4ec3afad39045ae9a8f969a9cbfe336 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:44.129873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:44.133698 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:44.248939 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def32ce-113d-4eb2-a2b7-1ae2adea9df8 service nova] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:44.259021 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101a0ba-d8bd-4d58-ba4c-a7f674a719fa admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:44.269473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:44.270319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:44.270650 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:44.271047 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e admin admin] PATH_INFO: `/v3/projects/ffb6e23c854e4ad394abb04963f2d896` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:44.273773 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffb6e23c854e4ad394abb04963f2d896)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:44.278363 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:44.394407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f99eb-e182-4ed6-b745-a74d7e75e2e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:44.395070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f99eb-e182-4ed6-b745-a74d7e75e2e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:44.395855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f99eb-e182-4ed6-b745-a74d7e75e2e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:44.395855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f99eb-e182-4ed6-b745-a74d7e75e2e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:44.397786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0b1f99eb-e182-4ed6-b745-a74d7e75e2e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:44.402821 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffb6e23c854e4ad394abb04963f2d896', 'request_id': 'req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.414477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffb6e23c854e4ad394abb04963f2d896', 'request_id': 'req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.429712 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffb6e23c854e4ad394abb04963f2d896', 'request_id': 'req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.439257 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:44.453313 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e admin admin] The token cache is being invalidate because project ffb6e23c854e4ad394abb04963f2d896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:44.454233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.456799 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.467686 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 919/1826] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:44 2022] DELETE /identity/v3/projects/ffb6e23c854e4ad394abb04963f2d896 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:44.468824 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:44.470495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:44.554664 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1f99eb-e182-4ed6-b745-a74d7e75e2e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:44.556498 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1f99eb-e182-4ed6-b745-a74d7e75e2e6 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:44.576552 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bda38-fdd1-45e5-a3e9-0c2539962a2e admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:44.579919 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 908/1827] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:44 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:44.582454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401df93-58a3-4220-9605-7a4d2f6f89b7 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:44.582600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea08b8-cbd0-4c37-83eb-cd66f0f8fd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:44.582717 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401df93-58a3-4220-9605-7a4d2f6f89b7 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:44.583024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401df93-58a3-4220-9605-7a4d2f6f89b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:44.583344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea08b8-cbd0-4c37-83eb-cd66f0f8fd9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:44.583698 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea08b8-cbd0-4c37-83eb-cd66f0f8fd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:44.584088 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bea08b8-cbd0-4c37-83eb-cd66f0f8fd9c admin admin] PATH_INFO: `/v3/users/ccd80e067214498396d7c1a0e5ce00bd` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:44.586055 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea08b8-cbd0-4c37-83eb-cd66f0f8fd9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccd80e067214498396d7c1a0e5ce00bd)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:44.595618 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bea08b8-cbd0-4c37-83eb-cd66f0f8fd9c admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:44.613190 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-0401df93-58a3-4220-9605-7a4d2f6f89b7 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:44.615236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0401df93-58a3-4220-9605-7a4d2f6f89b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:44.643436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 909/1828] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:44.647246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954c168-08e8-49f1-adce-e5eaaa4bee7b None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:44.647530 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954c168-08e8-49f1-adce-e5eaaa4bee7b None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:44.647803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954c168-08e8-49f1-adce-e5eaaa4bee7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:44.691675 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-c954c168-08e8-49f1-adce-e5eaaa4bee7b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:44.696437 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c954c168-08e8-49f1-adce-e5eaaa4bee7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:44.719787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 910/1829] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:44.724291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954c168-08e8-49f1-adce-e5eaaa4bee7b None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:44.731679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6bea08b8-cbd0-4c37-83eb-cd66f0f8fd9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccd80e067214498396d7c1a0e5ce00bd'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.754665 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6bea08b8-cbd0-4c37-83eb-cd66f0f8fd9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccd80e067214498396d7c1a0e5ce00bd'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.755350 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6bea08b8-cbd0-4c37-83eb-cd66f0f8fd9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccd80e067214498396d7c1a0e5ce00bd'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.764586 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c954c168-08e8-49f1-adce-e5eaaa4bee7b None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:44.791231 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a6d70-56da-4a00-8527-7f0d109c42c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:44.791869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a6d70-56da-4a00-8527-7f0d109c42c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:44.792157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a6d70-56da-4a00-8527-7f0d109c42c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:44.792413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a6d70-56da-4a00-8527-7f0d109c42c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:44.793232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a6d70-56da-4a00-8527-7f0d109c42c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:44.793549 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a6d70-56da-4a00-8527-7f0d109c42c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:44.793662 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6bea08b8-cbd0-4c37-83eb-cd66f0f8fd9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccd80e067214498396d7c1a0e5ce00bd'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.795112 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6bea08b8-cbd0-4c37-83eb-cd66f0f8fd9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccd80e067214498396d7c1a0e5ce00bd'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:44.795433 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a6d70-56da-4a00-8527-7f0d109c42c7 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:44.822058 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 920/1830] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:44 2022] DELETE /identity/v3/users/ccd80e067214498396d7c1a0e5ce00bd => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:44.823203 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea08b8-cbd0-4c37-83eb-cd66f0f8fd9c admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:44.827780 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 911/1831] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:44.828639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-6bea08b8-cbd0-4c37-83eb-cd66f0f8fd9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:44.830657 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a6d70-56da-4a00-8527-7f0d109c42c7 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:44.832746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c77a6d70-56da-4a00-8527-7f0d109c42c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:44.852260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a6d70-56da-4a00-8527-7f0d109c42c7 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:44.914640 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:44.944171 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d5964-03f6-4728-8fc5-17db816fb241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:44.946189 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d5964-03f6-4728-8fc5-17db816fb241 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:44.946682 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d5964-03f6-4728-8fc5-17db816fb241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:44.947134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d5964-03f6-4728-8fc5-17db816fb241 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:44.948203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d5964-03f6-4728-8fc5-17db816fb241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:44.956157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d5964-03f6-4728-8fc5-17db816fb241 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:44.971989 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bea08b8-cbd0-4c37-83eb-cd66f0f8fd9c admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:44.984893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f8c1f-8ac4-4298-833f-fedbc54fae7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:44.985538 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f8c1f-8ac4-4298-833f-fedbc54fae7b service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:44.985812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f8c1f-8ac4-4298-833f-fedbc54fae7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:44.986082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f8c1f-8ac4-4298-833f-fedbc54fae7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:44.995674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a13f8c1f-8ac4-4298-833f-fedbc54fae7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:45.010398 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 912/1832] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:44 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:45.016395 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d5964-03f6-4728-8fc5-17db816fb241 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.050651 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:45.149855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f8c1f-8ac4-4298-833f-fedbc54fae7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.155088 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f8c1f-8ac4-4298-833f-fedbc54fae7b service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.166988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d5964-03f6-4728-8fc5-17db816fb241 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.184723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 921/1833] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:44 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:45.186792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf076f-6399-42c7-82ed-e56db681d4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.187643 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf076f-6399-42c7-82ed-e56db681d4aa admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.188048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf076f-6399-42c7-82ed-e56db681d4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.188392 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf076f-6399-42c7-82ed-e56db681d4aa admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.189372 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bf076f-6399-42c7-82ed-e56db681d4aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.191721 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bf076f-6399-42c7-82ed-e56db681d4aa admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.238218 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 913/1834] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:45 2022] POST /identity/v3/users => generated 294 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:45.239055 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f8c1f-8ac4-4298-833f-fedbc54fae7b service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.254651 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f8c1f-8ac4-4298-833f-fedbc54fae7b service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.262446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d5cba-68ef-4e9d-8289-418761ce69f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.263241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d5cba-68ef-4e9d-8289-418761ce69f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.264128 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d5cba-68ef-4e9d-8289-418761ce69f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.264534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d5cba-68ef-4e9d-8289-418761ce69f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.278655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d5cba-68ef-4e9d-8289-418761ce69f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.283212 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51d5cba-68ef-4e9d-8289-418761ce69f6 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.340978 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 922/1835] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:45.345251 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf076f-6399-42c7-82ed-e56db681d4aa admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.362060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf076f-6399-42c7-82ed-e56db681d4aa admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.370069 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de50d3-ccbd-4e97-8641-105800cfb6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.371262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de50d3-ccbd-4e97-8641-105800cfb6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.371400 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de50d3-ccbd-4e97-8641-105800cfb6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.372624 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de50d3-ccbd-4e97-8641-105800cfb6c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.372624 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de50d3-ccbd-4e97-8641-105800cfb6c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.375794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de50d3-ccbd-4e97-8641-105800cfb6c7 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.398311 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d5cba-68ef-4e9d-8289-418761ce69f6 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.400118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 914/1836] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:45.414051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d5cba-68ef-4e9d-8289-418761ce69f6 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.427718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79efa3-d4b7-402f-b42d-53242e274df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.428675 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79efa3-d4b7-402f-b42d-53242e274df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.429119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79efa3-d4b7-402f-b42d-53242e274df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.429575 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79efa3-d4b7-402f-b42d-53242e274df1 admin admin] PATH_INFO: `/v3/projects/d68940106e9245d1aefa7c6b64436d61/users/d4f2d6cbec524c908fb0522d80433c48/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.463672 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79efa3-d4b7-402f-b42d-53242e274df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d68940106e9245d1aefa7c6b64436d61, user_id=d4f2d6cbec524c908fb0522d80433c48, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.466597 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79efa3-d4b7-402f-b42d-53242e274df1 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.478520 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de50d3-ccbd-4e97-8641-105800cfb6c7 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.480448 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-57de50d3-ccbd-4e97-8641-105800cfb6c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:45.486676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 923/1837] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:45 2022] PUT /identity/v3/projects/d68940106e9245d1aefa7c6b64436d61/users/d4f2d6cbec524c908fb0522d80433c48/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:45.499198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617ba9d-a55f-46a3-a20b-5c853cd17ebd None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.504574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de50d3-ccbd-4e97-8641-105800cfb6c7 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.506421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617ba9d-a55f-46a3-a20b-5c853cd17ebd None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.508140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617ba9d-a55f-46a3-a20b-5c853cd17ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.566621 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-0617ba9d-a55f-46a3-a20b-5c853cd17ebd None None] MFA Rules not processed for user `d4f2d6cbec524c908fb0522d80433c48`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:45.568065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-0617ba9d-a55f-46a3-a20b-5c853cd17ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:45.610424 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc9d5c-cbcc-484d-98e7-e2cfc6c0d101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.611126 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc9d5c-cbcc-484d-98e7-e2cfc6c0d101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.611767 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc9d5c-cbcc-484d-98e7-e2cfc6c0d101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.612078 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc9d5c-cbcc-484d-98e7-e2cfc6c0d101 admin admin] PATH_INFO: `/v3/projects/f2ff22a8aa6e4b7cb3be2ded8a76daa7` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.613438 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc9d5c-cbcc-484d-98e7-e2cfc6c0d101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2ff22a8aa6e4b7cb3be2ded8a76daa7)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.615840 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc9d5c-cbcc-484d-98e7-e2cfc6c0d101 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.649875 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 924/1838] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:45.654801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617ba9d-a55f-46a3-a20b-5c853cd17ebd None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.664435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:45.665827 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617ba9d-a55f-46a3-a20b-5c853cd17ebd None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.667817 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-17bc9d5c-cbcc-484d-98e7-e2cfc6c0d101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2ff22a8aa6e4b7cb3be2ded8a76daa7', 'request_id': 'req-17bc9d5c-cbcc-484d-98e7-e2cfc6c0d101'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:45.676011 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-17bc9d5c-cbcc-484d-98e7-e2cfc6c0d101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2ff22a8aa6e4b7cb3be2ded8a76daa7', 'request_id': 'req-17bc9d5c-cbcc-484d-98e7-e2cfc6c0d101'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:45.681708 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-17bc9d5c-cbcc-484d-98e7-e2cfc6c0d101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2ff22a8aa6e4b7cb3be2ded8a76daa7', 'request_id': 'req-17bc9d5c-cbcc-484d-98e7-e2cfc6c0d101'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:45.694892 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-17bc9d5c-cbcc-484d-98e7-e2cfc6c0d101 admin admin] The token cache is being invalidate because project f2ff22a8aa6e4b7cb3be2ded8a76daa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:45.695114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-17bc9d5c-cbcc-484d-98e7-e2cfc6c0d101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:45.695773 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-17bc9d5c-cbcc-484d-98e7-e2cfc6c0d101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:45.697303 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 915/1839] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:45 2022] DELETE /identity/v3/projects/f2ff22a8aa6e4b7cb3be2ded8a76daa7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:45.703124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61b22a-1d90-4340-9018-c0d584cca75b None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.703410 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61b22a-1d90-4340-9018-c0d584cca75b None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.703708 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61b22a-1d90-4340-9018-c0d584cca75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.724639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9648295-c1ad-4b0d-bc1f-4578163b3f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.725254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9648295-c1ad-4b0d-bc1f-4578163b3f01 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.725378 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-0a61b22a-1d90-4340-9018-c0d584cca75b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:45.725532 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9648295-c1ad-4b0d-bc1f-4578163b3f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.725801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9648295-c1ad-4b0d-bc1f-4578163b3f01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.726424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9648295-c1ad-4b0d-bc1f-4578163b3f01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.726561 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0a61b22a-1d90-4340-9018-c0d584cca75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:45.728353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9648295-c1ad-4b0d-bc1f-4578163b3f01 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.737015 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 916/1840] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:45.744771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d021d9-ac77-4487-90ba-da27e1f6e63e None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.745063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d021d9-ac77-4487-90ba-da27e1f6e63e None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.745568 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d021d9-ac77-4487-90ba-da27e1f6e63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.757241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 925/1841] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:45 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:45.761559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9648295-c1ad-4b0d-bc1f-4578163b3f01 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.763418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-f9648295-c1ad-4b0d-bc1f-4578163b3f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:45.767641 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-42d021d9-ac77-4487-90ba-da27e1f6e63e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:45.768809 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-42d021d9-ac77-4487-90ba-da27e1f6e63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:45.774901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:45.839426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9648295-c1ad-4b0d-bc1f-4578163b3f01 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.842604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 917/1842] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:45.844791 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9801cb9-d19e-435b-a4da-68f4dcb3f3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.845417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9801cb9-d19e-435b-a4da-68f4dcb3f3ec admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.845752 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9801cb9-d19e-435b-a4da-68f4dcb3f3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.846218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9801cb9-d19e-435b-a4da-68f4dcb3f3ec admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.846891 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9801cb9-d19e-435b-a4da-68f4dcb3f3ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.847018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d021d9-ac77-4487-90ba-da27e1f6e63e None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.848823 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9801cb9-d19e-435b-a4da-68f4dcb3f3ec admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.856697 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d021d9-ac77-4487-90ba-da27e1f6e63e None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.861986 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a05d7-a1a7-492d-a8a6-338e735e756e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.862182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a05d7-a1a7-492d-a8a6-338e735e756e admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.862459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a05d7-a1a7-492d-a8a6-338e735e756e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.862726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a05d7-a1a7-492d-a8a6-338e735e756e admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.863480 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a05d7-a1a7-492d-a8a6-338e735e756e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:45.863781 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a05d7-a1a7-492d-a8a6-338e735e756e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.866006 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a05d7-a1a7-492d-a8a6-338e735e756e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.875315 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 926/1843] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:45 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:45.879284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9801cb9-d19e-435b-a4da-68f4dcb3f3ec admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.883282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 918/1844] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:45.887788 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a05d7-a1a7-492d-a8a6-338e735e756e admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.894759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9801cb9-d19e-435b-a4da-68f4dcb3f3ec admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.899714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a05d7-a1a7-492d-a8a6-338e735e756e admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:45.902980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe7848-7793-4c21-9d4c-69100d277841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.903841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe7848-7793-4c21-9d4c-69100d277841 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.904186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe7848-7793-4c21-9d4c-69100d277841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.904556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe7848-7793-4c21-9d4c-69100d277841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.905279 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe7848-7793-4c21-9d4c-69100d277841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.905394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813ce28-8d3b-4e22-bdc6-64299e1c902d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:45.906043 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813ce28-8d3b-4e22-bdc6-64299e1c902d admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:45.906323 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813ce28-8d3b-4e22-bdc6-64299e1c902d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:45.906593 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813ce28-8d3b-4e22-bdc6-64299e1c902d admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:45.907309 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe7848-7793-4c21-9d4c-69100d277841 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.907626 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813ce28-8d3b-4e22-bdc6-64299e1c902d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:45.909913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813ce28-8d3b-4e22-bdc6-64299e1c902d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:45.935435 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 927/1845] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:45.940020 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe7848-7793-4c21-9d4c-69100d277841 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.940846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 919/1846] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:45 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:45.945283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813ce28-8d3b-4e22-bdc6-64299e1c902d admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:45.951515 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe7848-7793-4c21-9d4c-69100d277841 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.010169 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437d94c-5a82-4896-890e-8ec44b65afa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.010895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437d94c-5a82-4896-890e-8ec44b65afa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.010993 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813ce28-8d3b-4e22-bdc6-64299e1c902d admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.011089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437d94c-5a82-4896-890e-8ec44b65afa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.011349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437d94c-5a82-4896-890e-8ec44b65afa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.011967 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437d94c-5a82-4896-890e-8ec44b65afa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.013555 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437d94c-5a82-4896-890e-8ec44b65afa5 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.016938 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097728d8-27b1-4e60-aac6-636205f48842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.017571 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097728d8-27b1-4e60-aac6-636205f48842 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.017815 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097728d8-27b1-4e60-aac6-636205f48842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.018084 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097728d8-27b1-4e60-aac6-636205f48842 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.018725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097728d8-27b1-4e60-aac6-636205f48842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.021144 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097728d8-27b1-4e60-aac6-636205f48842 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.027331 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 928/1847] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:46.032108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437d94c-5a82-4896-890e-8ec44b65afa5 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.049773 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 920/1848] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:45 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:46.053950 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097728d8-27b1-4e60-aac6-636205f48842 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.055684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437d94c-5a82-4896-890e-8ec44b65afa5 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.062837 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae04da-990c-40eb-b0bf-7d4d7fb78e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.063570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae04da-990c-40eb-b0bf-7d4d7fb78e57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.063894 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae04da-990c-40eb-b0bf-7d4d7fb78e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.064229 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae04da-990c-40eb-b0bf-7d4d7fb78e57 admin admin] PATH_INFO: `/v3/projects/755be4b4fa1647a39b4f592331fe8702/users/31bc6032d2d248639b5fe6b518c0041f/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.064325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097728d8-27b1-4e60-aac6-636205f48842 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.071172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388e7de-ee28-4f50-946f-7f88fd4559df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.071782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388e7de-ee28-4f50-946f-7f88fd4559df admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.072063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388e7de-ee28-4f50-946f-7f88fd4559df admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.072334 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388e7de-ee28-4f50-946f-7f88fd4559df admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.072983 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388e7de-ee28-4f50-946f-7f88fd4559df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.074593 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388e7de-ee28-4f50-946f-7f88fd4559df admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.083335 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae04da-990c-40eb-b0bf-7d4d7fb78e57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=755be4b4fa1647a39b4f592331fe8702, user_id=31bc6032d2d248639b5fe6b518c0041f, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.085683 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 921/1849] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:46.086171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae04da-990c-40eb-b0bf-7d4d7fb78e57 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.089717 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388e7de-ee28-4f50-946f-7f88fd4559df admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.099690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388e7de-ee28-4f50-946f-7f88fd4559df admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.100672 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 929/1850] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:46 2022] PUT /identity/v3/projects/755be4b4fa1647a39b4f592331fe8702/users/31bc6032d2d248639b5fe6b518c0041f/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:46.107931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:46.111144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92b71a-a341-4a4b-bc5e-ea1abc063569 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.111433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92b71a-a341-4a4b-bc5e-ea1abc063569 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.111710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92b71a-a341-4a4b-bc5e-ea1abc063569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.146016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-fd92b71a-a341-4a4b-bc5e-ea1abc063569 None None] MFA Rules not processed for user `31bc6032d2d248639b5fe6b518c0041f`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:46.148444 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-fd92b71a-a341-4a4b-bc5e-ea1abc063569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:46.170094 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b2cea-a8f6-48f2-b4ee-041451304ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.170710 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b2cea-a8f6-48f2-b4ee-041451304ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.171019 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b2cea-a8f6-48f2-b4ee-041451304ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.171701 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b2cea-a8f6-48f2-b4ee-041451304ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.172358 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b2cea-a8f6-48f2-b4ee-041451304ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.174273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b2cea-a8f6-48f2-b4ee-041451304ed6 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.187257 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 922/1851] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:46.191401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b2cea-a8f6-48f2-b4ee-041451304ed6 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.202429 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b2cea-a8f6-48f2-b4ee-041451304ed6 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.208592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e9375-bcf1-433b-b453-7340652d2a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.209916 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e9375-bcf1-433b-b453-7340652d2a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.210255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e9375-bcf1-433b-b453-7340652d2a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.210597 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e9375-bcf1-433b-b453-7340652d2a1c admin admin] PATH_INFO: `/v3/projects/d02a1da1edb74420b35ae5a2bd680064/users/4cd00d160d824cc1852050a31eecc513/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.215840 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:46.229669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e9375-bcf1-433b-b453-7340652d2a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d02a1da1edb74420b35ae5a2bd680064, user_id=4cd00d160d824cc1852050a31eecc513, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.232586 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e9375-bcf1-433b-b453-7340652d2a1c admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.239332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 930/1852] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:46.243861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92b71a-a341-4a4b-bc5e-ea1abc063569 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.255467 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 923/1853] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:46 2022] PUT /identity/v3/projects/d02a1da1edb74420b35ae5a2bd680064/users/4cd00d160d824cc1852050a31eecc513/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:46.267063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d8708-a169-468e-a40f-6d915919f305 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.267333 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d8708-a169-468e-a40f-6d915919f305 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.267648 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d8708-a169-468e-a40f-6d915919f305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.271556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92b71a-a341-4a4b-bc5e-ea1abc063569 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.303337 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-b61d8708-a169-468e-a40f-6d915919f305 None None] MFA Rules not processed for user `4cd00d160d824cc1852050a31eecc513`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:46.304643 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b61d8708-a169-468e-a40f-6d915919f305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:46.361463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19f68c-18a4-43b3-9e28-73ae033e328f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.362451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19f68c-18a4-43b3-9e28-73ae033e328f admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.362968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19f68c-18a4-43b3-9e28-73ae033e328f admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.363425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19f68c-18a4-43b3-9e28-73ae033e328f admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.364425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19f68c-18a4-43b3-9e28-73ae033e328f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.367122 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19f68c-18a4-43b3-9e28-73ae033e328f admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.408734 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 931/1854] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:46 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:46.411546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19f68c-18a4-43b3-9e28-73ae033e328f admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.412041 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 924/1855] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:46.423858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d8708-a169-468e-a40f-6d915919f305 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.425342 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b61d8708-a169-468e-a40f-6d915919f305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:46.496115 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19f68c-18a4-43b3-9e28-73ae033e328f admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.502838 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740f903-9cdc-4e5e-a7c6-4422d3432dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.505433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740f903-9cdc-4e5e-a7c6-4422d3432dfb admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.505624 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740f903-9cdc-4e5e-a7c6-4422d3432dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.506409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740f903-9cdc-4e5e-a7c6-4422d3432dfb admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.507439 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740f903-9cdc-4e5e-a7c6-4422d3432dfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.509972 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d8708-a169-468e-a40f-6d915919f305 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.511224 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740f903-9cdc-4e5e-a7c6-4422d3432dfb admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.547552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:46.559061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 932/1856] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:46 2022] POST /identity/v3/users => generated 299 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:46.560522 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740f903-9cdc-4e5e-a7c6-4422d3432dfb admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.576219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740f903-9cdc-4e5e-a7c6-4422d3432dfb admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.579903 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d9b9f-90bd-45b3-bf54-bcbd9f747b8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.580531 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d9b9f-90bd-45b3-bf54-bcbd9f747b8a service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.580808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d9b9f-90bd-45b3-bf54-bcbd9f747b8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.581080 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d9b9f-90bd-45b3-bf54-bcbd9f747b8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.581892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef0f70-5c63-4b93-8b91-25fb6fc570a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.582640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef0f70-5c63-4b93-8b91-25fb6fc570a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.583056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef0f70-5c63-4b93-8b91-25fb6fc570a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.583436 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef0f70-5c63-4b93-8b91-25fb6fc570a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.584909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef0f70-5c63-4b93-8b91-25fb6fc570a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.586595 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef0f70-5c63-4b93-8b91-25fb6fc570a2 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.601253 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 933/1857] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:46.605482 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef0f70-5c63-4b93-8b91-25fb6fc570a2 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.623842 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef0f70-5c63-4b93-8b91-25fb6fc570a2 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.630100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ccb17-111a-4886-a544-328874a91669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.631422 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ccb17-111a-4886-a544-328874a91669 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.631648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ccb17-111a-4886-a544-328874a91669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.631924 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ccb17-111a-4886-a544-328874a91669 admin admin] PATH_INFO: `/v3/projects/50d7421b77504bf4a43ab6e1dc9f203e/users/f3481b76a9e34263b219cd0ec53ee8b3/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.648712 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d9b9f-90bd-45b3-bf54-bcbd9f747b8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.649194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ccb17-111a-4886-a544-328874a91669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50d7421b77504bf4a43ab6e1dc9f203e, user_id=f3481b76a9e34263b219cd0ec53ee8b3, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.650156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402d9b9f-90bd-45b3-bf54-bcbd9f747b8a service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.652286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ccb17-111a-4886-a544-328874a91669 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.661447 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 925/1858] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:46.667322 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 934/1859] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:46 2022] PUT /identity/v3/projects/50d7421b77504bf4a43ab6e1dc9f203e/users/f3481b76a9e34263b219cd0ec53ee8b3/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:46.672811 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d9b9f-90bd-45b3-bf54-bcbd9f747b8a service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.689108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d9b9f-90bd-45b3-bf54-bcbd9f747b8a service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.756542 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7315edc4-3c74-4583-9631-e1f1b7f3ae25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.757167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315edc4-3c74-4583-9631-e1f1b7f3ae25 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.757471 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315edc4-3c74-4583-9631-e1f1b7f3ae25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.757748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315edc4-3c74-4583-9631-e1f1b7f3ae25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.758394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7315edc4-3c74-4583-9631-e1f1b7f3ae25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.760034 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7315edc4-3c74-4583-9631-e1f1b7f3ae25 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.770250 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 926/1860] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:46.775377 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ccb17-111a-4886-a544-328874a91669 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.786422 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ccb17-111a-4886-a544-328874a91669 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:46.791194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723ea9d-edc7-43da-b537-fd3278ed7cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:46.791851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723ea9d-edc7-43da-b537-fd3278ed7cd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.792133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723ea9d-edc7-43da-b537-fd3278ed7cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.792431 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723ea9d-edc7-43da-b537-fd3278ed7cd9 admin admin] PATH_INFO: `/v3/projects/50d7421b77504bf4a43ab6e1dc9f203e/users/f3481b76a9e34263b219cd0ec53ee8b3/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.794352 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723ea9d-edc7-43da-b537-fd3278ed7cd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50d7421b77504bf4a43ab6e1dc9f203e, user_id=f3481b76a9e34263b219cd0ec53ee8b3, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:46.797056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723ea9d-edc7-43da-b537-fd3278ed7cd9 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:46.808553 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 935/1861] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:46 2022] PUT /identity/v3/projects/50d7421b77504bf4a43ab6e1dc9f203e/users/f3481b76a9e34263b219cd0ec53ee8b3/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:46.819900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf02298-7da3-4186-95ec-8f0d8a3e04ff None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:46.820972 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf02298-7da3-4186-95ec-8f0d8a3e04ff None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:46.821260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf02298-7da3-4186-95ec-8f0d8a3e04ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:46.852296 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-2bf02298-7da3-4186-95ec-8f0d8a3e04ff None None] MFA Rules not processed for user `f3481b76a9e34263b219cd0ec53ee8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:46.853796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2bf02298-7da3-4186-95ec-8f0d8a3e04ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:46.945849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 927/1862] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:46.954441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723ea9d-edc7-43da-b537-fd3278ed7cd9 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:46.978370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:47.027319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723ea9d-edc7-43da-b537-fd3278ed7cd9 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:47.031822 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdc3b2-78e4-4614-90d1-1a7eceb2871a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:47.032412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdc3b2-78e4-4614-90d1-1a7eceb2871a service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:47.032723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdc3b2-78e4-4614-90d1-1a7eceb2871a service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:47.032970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdc3b2-78e4-4614-90d1-1a7eceb2871a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:47.044943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbdc3b2-78e4-4614-90d1-1a7eceb2871a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:47.046381 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbdc3b2-78e4-4614-90d1-1a7eceb2871a service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:47.059957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 936/1863] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:47.623309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf02298-7da3-4186-95ec-8f0d8a3e04ff None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:47.624964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2bf02298-7da3-4186-95ec-8f0d8a3e04ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:47.633290 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:47.638023 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf02298-7da3-4186-95ec-8f0d8a3e04ff None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:47.642044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53197f51-3420-4d90-ac2b-b1588485610b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:47.642663 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53197f51-3420-4d90-ac2b-b1588485610b service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:47.642972 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53197f51-3420-4d90-ac2b-b1588485610b service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:47.643245 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53197f51-3420-4d90-ac2b-b1588485610b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:47.645153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-53197f51-3420-4d90-ac2b-b1588485610b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:47.732666 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53197f51-3420-4d90-ac2b-b1588485610b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:47.734113 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53197f51-3420-4d90-ac2b-b1588485610b service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:47.747116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 928/1864] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:51.589140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdc3b2-78e4-4614-90d1-1a7eceb2871a service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:51.590880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-acbdc3b2-78e4-4614-90d1-1a7eceb2871a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:51.656710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdc3b2-78e4-4614-90d1-1a7eceb2871a service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:51.661254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2a70f-d684-4ecc-ab85-8c528c031cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:51.662420 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2a70f-d684-4ecc-ab85-8c528c031cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:51.662698 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2a70f-d684-4ecc-ab85-8c528c031cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:51.663008 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2a70f-d684-4ecc-ab85-8c528c031cad admin admin] PATH_INFO: `/v3/users/24d4b3517d1b40eb87714fd7f56a3df5` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:51.664145 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2a70f-d684-4ecc-ab85-8c528c031cad admin admin] RBAC: Authorizing `identity:delete_user(user_id=24d4b3517d1b40eb87714fd7f56a3df5)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:51.666125 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2a70f-d684-4ecc-ab85-8c528c031cad admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:51.712577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d8c2a70f-d684-4ecc-ab85-8c528c031cad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24d4b3517d1b40eb87714fd7f56a3df5'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:51.721314 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d8c2a70f-d684-4ecc-ab85-8c528c031cad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24d4b3517d1b40eb87714fd7f56a3df5'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:51.721936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d8c2a70f-d684-4ecc-ab85-8c528c031cad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24d4b3517d1b40eb87714fd7f56a3df5'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:51.736063 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d8c2a70f-d684-4ecc-ab85-8c528c031cad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24d4b3517d1b40eb87714fd7f56a3df5'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:51.736762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d8c2a70f-d684-4ecc-ab85-8c528c031cad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24d4b3517d1b40eb87714fd7f56a3df5'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:51.749634 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 937/1865] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:51 2022] DELETE /identity/v3/users/24d4b3517d1b40eb87714fd7f56a3df5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:51.754326 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3227d-746e-4faf-9039-8757a3dd47c6 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:51.754875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3227d-746e-4faf-9039-8757a3dd47c6 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:51.754991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3227d-746e-4faf-9039-8757a3dd47c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:51.776709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-3fc3227d-746e-4faf-9039-8757a3dd47c6 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:51.778810 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3fc3227d-746e-4faf-9039-8757a3dd47c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:51.854932 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 929/1866] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:51.881943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2a70f-d684-4ecc-ab85-8c528c031cad admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:51.883426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d8c2a70f-d684-4ecc-ab85-8c528c031cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:51.940750 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:51.963226 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2a70f-d684-4ecc-ab85-8c528c031cad admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:51.963226 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fec6b4-5bc7-4bae-8761-acfe711e96e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:51.963226 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fec6b4-5bc7-4bae-8761-acfe711e96e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:51.963226 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fec6b4-5bc7-4bae-8761-acfe711e96e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:51.963541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fec6b4-5bc7-4bae-8761-acfe711e96e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:51.973351 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fec6b4-5bc7-4bae-8761-acfe711e96e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:51.975528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fec6b4-5bc7-4bae-8761-acfe711e96e3 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:51.985245 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 938/1867] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:52.292824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc3227d-746e-4faf-9039-8757a3dd47c6 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:52.294485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3fc3227d-746e-4faf-9039-8757a3dd47c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:52.307193 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc3227d-746e-4faf-9039-8757a3dd47c6 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:52.319796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0fce5-9772-4b28-821c-b918a3ca8dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:52.320544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0fce5-9772-4b28-821c-b918a3ca8dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:52.321031 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0fce5-9772-4b28-821c-b918a3ca8dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:52.321229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0fce5-9772-4b28-821c-b918a3ca8dab admin admin] PATH_INFO: `/v3/projects/1073a3c6ec9d4b96aaa7fc219b87343f` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:52.322244 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e0fce5-9772-4b28-821c-b918a3ca8dab admin admin] RBAC: Authorizing `identity:delete_project(project_id=1073a3c6ec9d4b96aaa7fc219b87343f)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:52.325205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e0fce5-9772-4b28-821c-b918a3ca8dab admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:52.384272 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-14e0fce5-9772-4b28-821c-b918a3ca8dab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1073a3c6ec9d4b96aaa7fc219b87343f', 'request_id': 'req-14e0fce5-9772-4b28-821c-b918a3ca8dab'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:52.391284 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-14e0fce5-9772-4b28-821c-b918a3ca8dab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1073a3c6ec9d4b96aaa7fc219b87343f', 'request_id': 'req-14e0fce5-9772-4b28-821c-b918a3ca8dab'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:52.398017 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-14e0fce5-9772-4b28-821c-b918a3ca8dab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1073a3c6ec9d4b96aaa7fc219b87343f', 'request_id': 'req-14e0fce5-9772-4b28-821c-b918a3ca8dab'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:52.412574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-14e0fce5-9772-4b28-821c-b918a3ca8dab admin admin] The token cache is being invalidate because project 1073a3c6ec9d4b96aaa7fc219b87343f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:52.412861 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-14e0fce5-9772-4b28-821c-b918a3ca8dab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:52.413493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-14e0fce5-9772-4b28-821c-b918a3ca8dab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:52.417986 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 930/1868] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:52 2022] DELETE /identity/v3/projects/1073a3c6ec9d4b96aaa7fc219b87343f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:52.419584 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fec6b4-5bc7-4bae-8761-acfe711e96e3 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:52.430079 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fec6b4-5bc7-4bae-8761-acfe711e96e3 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:52.495149 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64b712-e897-40ec-8cbb-4e36241d6567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:52.495769 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64b712-e897-40ec-8cbb-4e36241d6567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:52.496044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64b712-e897-40ec-8cbb-4e36241d6567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:52.496311 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64b712-e897-40ec-8cbb-4e36241d6567 admin admin] PATH_INFO: `/v3/users/ea2d7b5b58584822b056ecf5f94df16e` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:52.497425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64b712-e897-40ec-8cbb-4e36241d6567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea2d7b5b58584822b056ecf5f94df16e)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:52.499414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64b712-e897-40ec-8cbb-4e36241d6567 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:52.547031 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-7b64b712-e897-40ec-8cbb-4e36241d6567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea2d7b5b58584822b056ecf5f94df16e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:52.555465 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-7b64b712-e897-40ec-8cbb-4e36241d6567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea2d7b5b58584822b056ecf5f94df16e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:52.556095 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-7b64b712-e897-40ec-8cbb-4e36241d6567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea2d7b5b58584822b056ecf5f94df16e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:52.568444 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-7b64b712-e897-40ec-8cbb-4e36241d6567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea2d7b5b58584822b056ecf5f94df16e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:52.569092 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-7b64b712-e897-40ec-8cbb-4e36241d6567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea2d7b5b58584822b056ecf5f94df16e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:52.581373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 939/1869] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:52 2022] DELETE /identity/v3/users/ea2d7b5b58584822b056ecf5f94df16e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:52.859054 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0fce5-9772-4b28-821c-b918a3ca8dab admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:52.860583 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-14e0fce5-9772-4b28-821c-b918a3ca8dab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:52.875330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0fce5-9772-4b28-821c-b918a3ca8dab admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:52.901548 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:52.935806 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55c0b6-75b4-4dce-b2d9-9f90d7481b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:52.936376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55c0b6-75b4-4dce-b2d9-9f90d7481b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:52.936651 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55c0b6-75b4-4dce-b2d9-9f90d7481b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:52.936919 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55c0b6-75b4-4dce-b2d9-9f90d7481b71 admin admin] PATH_INFO: `/v3/projects/77b7fe6b7e694e2583d754ed11ecb716` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:52.938035 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55c0b6-75b4-4dce-b2d9-9f90d7481b71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77b7fe6b7e694e2583d754ed11ecb716)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:52.939984 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55c0b6-75b4-4dce-b2d9-9f90d7481b71 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:52.989697 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ab55c0b6-75b4-4dce-b2d9-9f90d7481b71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77b7fe6b7e694e2583d754ed11ecb716', 'request_id': 'req-ab55c0b6-75b4-4dce-b2d9-9f90d7481b71'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:52.996084 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ab55c0b6-75b4-4dce-b2d9-9f90d7481b71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77b7fe6b7e694e2583d754ed11ecb716', 'request_id': 'req-ab55c0b6-75b4-4dce-b2d9-9f90d7481b71'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:53.002348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ab55c0b6-75b4-4dce-b2d9-9f90d7481b71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77b7fe6b7e694e2583d754ed11ecb716', 'request_id': 'req-ab55c0b6-75b4-4dce-b2d9-9f90d7481b71'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:53.017094 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ab55c0b6-75b4-4dce-b2d9-9f90d7481b71 admin admin] The token cache is being invalidate because project 77b7fe6b7e694e2583d754ed11ecb716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:53.017385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ab55c0b6-75b4-4dce-b2d9-9f90d7481b71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:53.018121 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ab55c0b6-75b4-4dce-b2d9-9f90d7481b71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:53.019613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 931/1870] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:52 2022] DELETE /identity/v3/projects/77b7fe6b7e694e2583d754ed11ecb716 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:53.025127 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a33e2-c4cb-4b4d-8574-529df6d81852 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.025606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a33e2-c4cb-4b4d-8574-529df6d81852 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.025706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a33e2-c4cb-4b4d-8574-529df6d81852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.047627 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-120a33e2-c4cb-4b4d-8574-529df6d81852 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:53.048868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-120a33e2-c4cb-4b4d-8574-529df6d81852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:53.103144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:53.119899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 940/1871] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:53.121668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6766ff6-987b-4fbc-b1fe-755d8c4254fb None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.121955 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6766ff6-987b-4fbc-b1fe-755d8c4254fb None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.122232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6766ff6-987b-4fbc-b1fe-755d8c4254fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.148145 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-d6766ff6-987b-4fbc-b1fe-755d8c4254fb None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:53.149488 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d6766ff6-987b-4fbc-b1fe-755d8c4254fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:53.158549 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 932/1872] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:53.163181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a33e2-c4cb-4b4d-8574-529df6d81852 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.174393 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a33e2-c4cb-4b4d-8574-529df6d81852 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.179540 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211540f-c324-4a30-a942-ea1f81cef6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.180158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211540f-c324-4a30-a942-ea1f81cef6fe admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.180433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211540f-c324-4a30-a942-ea1f81cef6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.180703 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211540f-c324-4a30-a942-ea1f81cef6fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.181440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211540f-c324-4a30-a942-ea1f81cef6fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:53.181643 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211540f-c324-4a30-a942-ea1f81cef6fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.183341 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211540f-c324-4a30-a942-ea1f81cef6fe admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.197422 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 941/1873] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:58:53.201202 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6766ff6-987b-4fbc-b1fe-755d8c4254fb None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.211427 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6766ff6-987b-4fbc-b1fe-755d8c4254fb None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.222761 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c0449-d47a-4dd4-9367-c3d122032f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.223378 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c0449-d47a-4dd4-9367-c3d122032f2e admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.223656 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c0449-d47a-4dd4-9367-c3d122032f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.223927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c0449-d47a-4dd4-9367-c3d122032f2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.224692 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c0449-d47a-4dd4-9367-c3d122032f2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.226579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c0449-d47a-4dd4-9367-c3d122032f2e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.254772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 933/1874] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:53 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:53.259509 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211540f-c324-4a30-a942-ea1f81cef6fe admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.270753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211540f-c324-4a30-a942-ea1f81cef6fe admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.337761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f14cd1-43e7-476e-b657-92fa7f59eb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.337975 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f14cd1-43e7-476e-b657-92fa7f59eb24 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.338222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f14cd1-43e7-476e-b657-92fa7f59eb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.338496 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f14cd1-43e7-476e-b657-92fa7f59eb24 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.339197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f14cd1-43e7-476e-b657-92fa7f59eb24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.341387 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f14cd1-43e7-476e-b657-92fa7f59eb24 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.364118 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 942/1875] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:53 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:53.369085 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c0449-d47a-4dd4-9367-c3d122032f2e admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.381231 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c0449-d47a-4dd4-9367-c3d122032f2e admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.386101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cda69d-00f1-4d7e-b95c-935ae6a6c016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.386874 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cda69d-00f1-4d7e-b95c-935ae6a6c016 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.387363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cda69d-00f1-4d7e-b95c-935ae6a6c016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.387641 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cda69d-00f1-4d7e-b95c-935ae6a6c016 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.388296 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cda69d-00f1-4d7e-b95c-935ae6a6c016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.389934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cda69d-00f1-4d7e-b95c-935ae6a6c016 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.404734 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 934/1876] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:53.406271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f14cd1-43e7-476e-b657-92fa7f59eb24 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.416335 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f14cd1-43e7-476e-b657-92fa7f59eb24 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.421098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02eab5c-52af-48fe-8478-73b487830268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.421806 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02eab5c-52af-48fe-8478-73b487830268 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.422154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02eab5c-52af-48fe-8478-73b487830268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.422494 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02eab5c-52af-48fe-8478-73b487830268 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.423248 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02eab5c-52af-48fe-8478-73b487830268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.424911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02eab5c-52af-48fe-8478-73b487830268 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.436561 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 943/1877] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:53.440736 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cda69d-00f1-4d7e-b95c-935ae6a6c016 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.451405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cda69d-00f1-4d7e-b95c-935ae6a6c016 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.465455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20efe39-bfbc-4884-830e-eab62964df26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.466134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20efe39-bfbc-4884-830e-eab62964df26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.467079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20efe39-bfbc-4884-830e-eab62964df26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.467288 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20efe39-bfbc-4884-830e-eab62964df26 admin admin] PATH_INFO: `/v3/projects/80320cebc6624a18b54bb683d7dabab0/users/2f06cb08ff8c4100b82f8566d2e28927/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.485795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20efe39-bfbc-4884-830e-eab62964df26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80320cebc6624a18b54bb683d7dabab0, user_id=2f06cb08ff8c4100b82f8566d2e28927, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.488866 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20efe39-bfbc-4884-830e-eab62964df26 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.504784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 935/1878] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:53 2022] PUT /identity/v3/projects/80320cebc6624a18b54bb683d7dabab0/users/2f06cb08ff8c4100b82f8566d2e28927/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:53.515836 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2203c51-4eb0-4813-aee3-aa7b0eb4d71a None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.516121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2203c51-4eb0-4813-aee3-aa7b0eb4d71a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.516394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2203c51-4eb0-4813-aee3-aa7b0eb4d71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.544930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-c2203c51-4eb0-4813-aee3-aa7b0eb4d71a None None] MFA Rules not processed for user `2f06cb08ff8c4100b82f8566d2e28927`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:53.546141 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-c2203c51-4eb0-4813-aee3-aa7b0eb4d71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:53.639003 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 944/1879] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:53.639159 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20efe39-bfbc-4884-830e-eab62964df26 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.689606 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:53.703851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20efe39-bfbc-4884-830e-eab62964df26 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.709922 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff5329-90ca-4793-80ed-1792c53f913c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.710580 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff5329-90ca-4793-80ed-1792c53f913c admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.710899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff5329-90ca-4793-80ed-1792c53f913c admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.711176 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff5329-90ca-4793-80ed-1792c53f913c admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.711813 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff5329-90ca-4793-80ed-1792c53f913c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.713721 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aff5329-90ca-4793-80ed-1792c53f913c admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.742772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 936/1880] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:53 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:53.747185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2203c51-4eb0-4813-aee3-aa7b0eb4d71a None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.757881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2203c51-4eb0-4813-aee3-aa7b0eb4d71a None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.809585 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:53.818035 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83f7aa-4be1-4004-838a-531a1af5f0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.818668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83f7aa-4be1-4004-838a-531a1af5f0f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.818996 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83f7aa-4be1-4004-838a-531a1af5f0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.819274 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83f7aa-4be1-4004-838a-531a1af5f0f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.819924 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83f7aa-4be1-4004-838a-531a1af5f0f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.821854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83f7aa-4be1-4004-838a-531a1af5f0f3 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.844718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 945/1881] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:53 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:53.848563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff5329-90ca-4793-80ed-1792c53f913c admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.865801 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff5329-90ca-4793-80ed-1792c53f913c admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.879568 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a0f69-ccb0-4203-8c32-732fce592aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.880603 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a0f69-ccb0-4203-8c32-732fce592aaa admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.881154 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a0f69-ccb0-4203-8c32-732fce592aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.881562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a0f69-ccb0-4203-8c32-732fce592aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.882676 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a0f69-ccb0-4203-8c32-732fce592aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.884785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a0f69-ccb0-4203-8c32-732fce592aaa admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.898588 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 937/1882] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:53.903024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83f7aa-4be1-4004-838a-531a1af5f0f3 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:53.915375 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83f7aa-4be1-4004-838a-531a1af5f0f3 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:53.920747 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f07f7-08d3-477c-9b6f-04d83c53cdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:53.921947 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f07f7-08d3-477c-9b6f-04d83c53cdcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:53.922491 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f07f7-08d3-477c-9b6f-04d83c53cdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:53.923105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f07f7-08d3-477c-9b6f-04d83c53cdcf admin admin] PATH_INFO: `/v3/projects/1f021e1ce06a41a494fa261156b871e3/users/aa8cd9e2d8c447d5a598f98cacd536eb/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:53.947560 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f07f7-08d3-477c-9b6f-04d83c53cdcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f021e1ce06a41a494fa261156b871e3, user_id=aa8cd9e2d8c447d5a598f98cacd536eb, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:53.950848 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f07f7-08d3-477c-9b6f-04d83c53cdcf admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:53.983589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 946/1883] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:53 2022] PUT /identity/v3/projects/1f021e1ce06a41a494fa261156b871e3/users/aa8cd9e2d8c447d5a598f98cacd536eb/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:53.984502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a0f69-ccb0-4203-8c32-732fce592aaa admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.005025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a0f69-ccb0-4203-8c32-732fce592aaa admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.093633 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b00ad-9ba2-49a8-9488-33ed223fe381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.094351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b00ad-9ba2-49a8-9488-33ed223fe381 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.094688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b00ad-9ba2-49a8-9488-33ed223fe381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.095187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b00ad-9ba2-49a8-9488-33ed223fe381 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.095651 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b00ad-9ba2-49a8-9488-33ed223fe381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:54.097365 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b00ad-9ba2-49a8-9488-33ed223fe381 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:54.115881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f07f7-08d3-477c-9b6f-04d83c53cdcf admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.118188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 938/1884] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:54.128630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f07f7-08d3-477c-9b6f-04d83c53cdcf admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.134507 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff088aeb-d1f6-47f2-9ce6-2d671f93b5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.135689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff088aeb-d1f6-47f2-9ce6-2d671f93b5a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.136228 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff088aeb-d1f6-47f2-9ce6-2d671f93b5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.136699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff088aeb-d1f6-47f2-9ce6-2d671f93b5a1 admin admin] PATH_INFO: `/v3/projects/1f021e1ce06a41a494fa261156b871e3/users/aa8cd9e2d8c447d5a598f98cacd536eb/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.139321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff088aeb-d1f6-47f2-9ce6-2d671f93b5a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f021e1ce06a41a494fa261156b871e3, user_id=aa8cd9e2d8c447d5a598f98cacd536eb, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:54.142888 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff088aeb-d1f6-47f2-9ce6-2d671f93b5a1 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:54.156551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 947/1885] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:54 2022] PUT /identity/v3/projects/1f021e1ce06a41a494fa261156b871e3/users/aa8cd9e2d8c447d5a598f98cacd536eb/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:54.169667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622f90f-243f-4c08-bac5-6e2a49ed5581 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.169918 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622f90f-243f-4c08-bac5-6e2a49ed5581 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.170258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622f90f-243f-4c08-bac5-6e2a49ed5581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.203937 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-c622f90f-243f-4c08-bac5-6e2a49ed5581 None None] MFA Rules not processed for user `aa8cd9e2d8c447d5a598f98cacd536eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:54.205448 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c622f90f-243f-4c08-bac5-6e2a49ed5581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:54.421021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 939/1886] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:54.492752 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff088aeb-d1f6-47f2-9ce6-2d671f93b5a1 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.494873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ff088aeb-d1f6-47f2-9ce6-2d671f93b5a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:54.606873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff088aeb-d1f6-47f2-9ce6-2d671f93b5a1 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.614822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622f90f-243f-4c08-bac5-6e2a49ed5581 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.615969 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386120cb-582e-4572-85a7-c56d6eff6eb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.616572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c622f90f-243f-4c08-bac5-6e2a49ed5581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:54.616781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386120cb-582e-4572-85a7-c56d6eff6eb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.617134 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386120cb-582e-4572-85a7-c56d6eff6eb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.617478 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386120cb-582e-4572-85a7-c56d6eff6eb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.648341 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386120cb-582e-4572-85a7-c56d6eff6eb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:54.650479 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386120cb-582e-4572-85a7-c56d6eff6eb9 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:54.674508 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 948/1887] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:54.722493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:54.747237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622f90f-243f-4c08-bac5-6e2a49ed5581 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.756936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1456f32-abba-493a-b99b-f345bd7628f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.757667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1456f32-abba-493a-b99b-f345bd7628f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.758007 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1456f32-abba-493a-b99b-f345bd7628f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.758339 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1456f32-abba-493a-b99b-f345bd7628f0 admin admin] PATH_INFO: `/v3/users/66b1c53f7fb140a8a4cd7ec7c26bc899` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.759729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1456f32-abba-493a-b99b-f345bd7628f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66b1c53f7fb140a8a4cd7ec7c26bc899)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:54.762638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1456f32-abba-493a-b99b-f345bd7628f0 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:54.832638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-d1456f32-abba-493a-b99b-f345bd7628f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66b1c53f7fb140a8a4cd7ec7c26bc899'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:54.850516 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-d1456f32-abba-493a-b99b-f345bd7628f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66b1c53f7fb140a8a4cd7ec7c26bc899'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:54.866617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-d1456f32-abba-493a-b99b-f345bd7628f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66b1c53f7fb140a8a4cd7ec7c26bc899'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:54.885142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386120cb-582e-4572-85a7-c56d6eff6eb9 service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:54.895105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386120cb-582e-4572-85a7-c56d6eff6eb9 service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:54.895920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-d1456f32-abba-493a-b99b-f345bd7628f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66b1c53f7fb140a8a4cd7ec7c26bc899'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:54.900882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-d1456f32-abba-493a-b99b-f345bd7628f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66b1c53f7fb140a8a4cd7ec7c26bc899'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:54.903749 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b04061-16b6-4337-b594-86b0a1fbc6c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:54.904374 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b04061-16b6-4337-b594-86b0a1fbc6c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.904653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b04061-16b6-4337-b594-86b0a1fbc6c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.904922 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b04061-16b6-4337-b594-86b0a1fbc6c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.907810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a3b04061-16b6-4337-b594-86b0a1fbc6c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:54.908701 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 940/1888] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:54 2022] DELETE /identity/v3/users/66b1c53f7fb140a8a4cd7ec7c26bc899 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:54.913597 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617f25a-774f-47c8-9add-faff75c1a819 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:54.913881 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617f25a-774f-47c8-9add-faff75c1a819 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:54.914156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617f25a-774f-47c8-9add-faff75c1a819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:54.935598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-b617f25a-774f-47c8-9add-faff75c1a819 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:54.936818 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b617f25a-774f-47c8-9add-faff75c1a819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:54.999059 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b04061-16b6-4337-b594-86b0a1fbc6c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:55.000725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b04061-16b6-4337-b594-86b0a1fbc6c8 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:55.012773 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 949/1889] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:55.013674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b04061-16b6-4337-b594-86b0a1fbc6c8 service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:55.015137 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a3b04061-16b6-4337-b594-86b0a1fbc6c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:55.027002 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b04061-16b6-4337-b594-86b0a1fbc6c8 service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:55.076919 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 941/1890] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:55.087635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617f25a-774f-47c8-9add-faff75c1a819 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:55.090127 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b617f25a-774f-47c8-9add-faff75c1a819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:55.103624 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:55.110613 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b4e9a-6bff-445a-84e5-b0c7c5f35610 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:55.111643 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b4e9a-6bff-445a-84e5-b0c7c5f35610 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:55.112301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b4e9a-6bff-445a-84e5-b0c7c5f35610 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:55.112921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b4e9a-6bff-445a-84e5-b0c7c5f35610 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:55.117023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-0c3b4e9a-6bff-445a-84e5-b0c7c5f35610 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:55.166923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617f25a-774f-47c8-9add-faff75c1a819 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:55.170133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a5ad3-4060-467b-966a-f249ab323036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:55.170788 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a5ad3-4060-467b-966a-f249ab323036 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:55.171067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a5ad3-4060-467b-966a-f249ab323036 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:55.171338 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a5ad3-4060-467b-966a-f249ab323036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:55.183882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a5ad3-4060-467b-966a-f249ab323036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:55.185727 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a5ad3-4060-467b-966a-f249ab323036 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:55.199139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 942/1891] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:55.205654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b4e9a-6bff-445a-84e5-b0c7c5f35610 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:55.207148 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b4e9a-6bff-445a-84e5-b0c7c5f35610 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:55.218414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 950/1892] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:55 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:55.436275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a5ad3-4060-467b-966a-f249ab323036 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:55.438045 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-652a5ad3-4060-467b-966a-f249ab323036 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:55.454279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a5ad3-4060-467b-966a-f249ab323036 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:55.459965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6f4f4-84da-4989-9f47-045b199d53ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:55.460712 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6f4f4-84da-4989-9f47-045b199d53ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:55.460983 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6f4f4-84da-4989-9f47-045b199d53ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:55.461256 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6f4f4-84da-4989-9f47-045b199d53ec admin admin] PATH_INFO: `/v3/users/4cd00d160d824cc1852050a31eecc513` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:55.462855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6f4f4-84da-4989-9f47-045b199d53ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cd00d160d824cc1852050a31eecc513)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:55.465132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6f4f4-84da-4989-9f47-045b199d53ec admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:55.470575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:55.568912 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6fa6f4f4-84da-4989-9f47-045b199d53ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cd00d160d824cc1852050a31eecc513'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:55.581528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6fa6f4f4-84da-4989-9f47-045b199d53ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cd00d160d824cc1852050a31eecc513'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:55.593886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6fa6f4f4-84da-4989-9f47-045b199d53ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cd00d160d824cc1852050a31eecc513'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:55.596014 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b4e9a-6bff-445a-84e5-b0c7c5f35610 service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:55.598113 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-0c3b4e9a-6bff-445a-84e5-b0c7c5f35610 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:55.620656 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6fa6f4f4-84da-4989-9f47-045b199d53ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cd00d160d824cc1852050a31eecc513'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:55.620891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6fa6f4f4-84da-4989-9f47-045b199d53ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cd00d160d824cc1852050a31eecc513'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:55.627888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 943/1893] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:55 2022] DELETE /identity/v3/users/4cd00d160d824cc1852050a31eecc513 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:55.628881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b4e9a-6bff-445a-84e5-b0c7c5f35610 service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:55.634572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9894a-95ce-4a21-8033-dc434f50310a None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:55.634931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9894a-95ce-4a21-8033-dc434f50310a None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:55.635315 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9894a-95ce-4a21-8033-dc434f50310a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:55.662265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-88d9894a-95ce-4a21-8033-dc434f50310a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:55.663538 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-88d9894a-95ce-4a21-8033-dc434f50310a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:55.716851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680e2db-0568-4762-b58a-9d95d6021034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:55.717922 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680e2db-0568-4762-b58a-9d95d6021034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:55.718398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680e2db-0568-4762-b58a-9d95d6021034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:55.718847 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680e2db-0568-4762-b58a-9d95d6021034 admin admin] PATH_INFO: `/v3/projects/57438b15417a4566bac79b68ce36c670` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:55.720944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c680e2db-0568-4762-b58a-9d95d6021034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57438b15417a4566bac79b68ce36c670)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:55.723280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c680e2db-0568-4762-b58a-9d95d6021034 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:55.748476 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 944/1894] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:55.760122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9894a-95ce-4a21-8033-dc434f50310a None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:55.762265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-88d9894a-95ce-4a21-8033-dc434f50310a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:55.807562 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c680e2db-0568-4762-b58a-9d95d6021034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57438b15417a4566bac79b68ce36c670', 'request_id': 'req-c680e2db-0568-4762-b58a-9d95d6021034'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:55.825188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c680e2db-0568-4762-b58a-9d95d6021034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57438b15417a4566bac79b68ce36c670', 'request_id': 'req-c680e2db-0568-4762-b58a-9d95d6021034'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:55.840408 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c680e2db-0568-4762-b58a-9d95d6021034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57438b15417a4566bac79b68ce36c670', 'request_id': 'req-c680e2db-0568-4762-b58a-9d95d6021034'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:55.865343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c680e2db-0568-4762-b58a-9d95d6021034 admin admin] The token cache is being invalidate because project 57438b15417a4566bac79b68ce36c670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:55.865999 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c680e2db-0568-4762-b58a-9d95d6021034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:55.866867 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c680e2db-0568-4762-b58a-9d95d6021034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:55.872377 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 951/1895] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:55 2022] DELETE /identity/v3/projects/57438b15417a4566bac79b68ce36c670 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:55.881134 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680e2db-0568-4762-b58a-9d95d6021034 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:55.885868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-c680e2db-0568-4762-b58a-9d95d6021034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:55.891082 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9894a-95ce-4a21-8033-dc434f50310a None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:55.950577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:55.962437 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:55.993211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222da80-7500-454d-b742-045f232a5482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:55.993851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222da80-7500-454d-b742-045f232a5482 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:55.994135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222da80-7500-454d-b742-045f232a5482 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:55.994409 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222da80-7500-454d-b742-045f232a5482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:56.003679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7222da80-7500-454d-b742-045f232a5482 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:56.031918 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680e2db-0568-4762-b58a-9d95d6021034 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:56.046761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea268c25-caf2-4ee0-a371-36698916ef3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:56.048616 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea268c25-caf2-4ee0-a371-36698916ef3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:56.048899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea268c25-caf2-4ee0-a371-36698916ef3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:56.049180 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea268c25-caf2-4ee0-a371-36698916ef3c admin admin] PATH_INFO: `/v3/users/77ed8dbff7824bd0b991816ada191a52` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:56.056742 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea268c25-caf2-4ee0-a371-36698916ef3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=77ed8dbff7824bd0b991816ada191a52)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:56.059351 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea268c25-caf2-4ee0-a371-36698916ef3c admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:56.060152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222da80-7500-454d-b742-045f232a5482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:56.063601 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222da80-7500-454d-b742-045f232a5482 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:56.094637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 945/1896] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:56.171761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ea268c25-caf2-4ee0-a371-36698916ef3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77ed8dbff7824bd0b991816ada191a52'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:56.189655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ea268c25-caf2-4ee0-a371-36698916ef3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77ed8dbff7824bd0b991816ada191a52'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:56.190885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ea268c25-caf2-4ee0-a371-36698916ef3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77ed8dbff7824bd0b991816ada191a52'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:56.216036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ea268c25-caf2-4ee0-a371-36698916ef3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77ed8dbff7824bd0b991816ada191a52'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:56.216860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ea268c25-caf2-4ee0-a371-36698916ef3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77ed8dbff7824bd0b991816ada191a52'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:56.242592 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 952/1897] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:55 2022] DELETE /identity/v3/users/77ed8dbff7824bd0b991816ada191a52 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:56.355768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3c0b0-ec90-43cf-910b-70340dcdcd31 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:56.356222 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3c0b0-ec90-43cf-910b-70340dcdcd31 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:56.356527 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3c0b0-ec90-43cf-910b-70340dcdcd31 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:56.358181 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 946/1898] 10.222.0.26 () {54 vars in 921 bytes} [Thu Sep 22 14:58:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 14:58:56.373545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a16aa-c126-4877-9e4a-0dd00e513a7d None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:56.373972 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a16aa-c126-4877-9e4a-0dd00e513a7d None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:56.374256 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a16aa-c126-4877-9e4a-0dd00e513a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:56.409955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-fc0a16aa-c126-4877-9e4a-0dd00e513a7d None None] MFA Rules not processed for user `0c9be41c39494e54a14398a1f03c4d69`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:56.413802 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-fc0a16aa-c126-4877-9e4a-0dd00e513a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:56.613902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 953/1899] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Sep 22 14:58:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:56.717745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3c0b0-ec90-43cf-910b-70340dcdcd31 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:56.719652 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e2f3c0b0-ec90-43cf-910b-70340dcdcd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:56.745205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3c0b0-ec90-43cf-910b-70340dcdcd31 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:56.752157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bee59-e4d8-470f-8f25-a43c8207c31a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:56.753298 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bee59-e4d8-470f-8f25-a43c8207c31a service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:56.753514 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bee59-e4d8-470f-8f25-a43c8207c31a service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:56.754230 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bee59-e4d8-470f-8f25-a43c8207c31a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:56.779045 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bee59-e4d8-470f-8f25-a43c8207c31a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:56.784922 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bee59-e4d8-470f-8f25-a43c8207c31a service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:56.823226 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 947/1900] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:57.007215 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a16aa-c126-4877-9e4a-0dd00e513a7d None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:57.009035 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-fc0a16aa-c126-4877-9e4a-0dd00e513a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:57.092169 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:57.122774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bee59-e4d8-470f-8f25-a43c8207c31a service nova] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:57.125051 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-ed1bee59-e4d8-470f-8f25-a43c8207c31a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:57.129945 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a16aa-c126-4877-9e4a-0dd00e513a7d None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:57.144817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1539b9f-4b4f-4ec8-bdc6-55a768a1b97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:57.145562 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1539b9f-4b4f-4ec8-bdc6-55a768a1b97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:57.145922 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1539b9f-4b4f-4ec8-bdc6-55a768a1b97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:57.146275 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1539b9f-4b4f-4ec8-bdc6-55a768a1b97b admin admin] PATH_INFO: `/v3/projects/234344d52dfd4d578fa9afef009c3713` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:57.149404 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1539b9f-4b4f-4ec8-bdc6-55a768a1b97b admin admin] RBAC: Authorizing `identity:delete_project(project_id=234344d52dfd4d578fa9afef009c3713)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:57.151461 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1539b9f-4b4f-4ec8-bdc6-55a768a1b97b admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:57.153389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bee59-e4d8-470f-8f25-a43c8207c31a service nova] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:57.171515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff1db4-6eba-4626-8f16-a66f8e636ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:57.174123 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff1db4-6eba-4626-8f16-a66f8e636ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:57.176956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff1db4-6eba-4626-8f16-a66f8e636ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:57.178706 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff1db4-6eba-4626-8f16-a66f8e636ce0 admin admin] PATH_INFO: `/v3/projects/d02a1da1edb74420b35ae5a2bd680064` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:57.182915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff1db4-6eba-4626-8f16-a66f8e636ce0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d02a1da1edb74420b35ae5a2bd680064)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:57.186421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff1db4-6eba-4626-8f16-a66f8e636ce0 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:57.261688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d1539b9f-4b4f-4ec8-bdc6-55a768a1b97b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '234344d52dfd4d578fa9afef009c3713', 'request_id': 'req-d1539b9f-4b4f-4ec8-bdc6-55a768a1b97b'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:57.263250 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-49ff1db4-6eba-4626-8f16-a66f8e636ce0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd02a1da1edb74420b35ae5a2bd680064', 'request_id': 'req-49ff1db4-6eba-4626-8f16-a66f8e636ce0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:57.278350 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-49ff1db4-6eba-4626-8f16-a66f8e636ce0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd02a1da1edb74420b35ae5a2bd680064', 'request_id': 'req-49ff1db4-6eba-4626-8f16-a66f8e636ce0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:57.279281 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d1539b9f-4b4f-4ec8-bdc6-55a768a1b97b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '234344d52dfd4d578fa9afef009c3713', 'request_id': 'req-d1539b9f-4b4f-4ec8-bdc6-55a768a1b97b'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:57.291062 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d1539b9f-4b4f-4ec8-bdc6-55a768a1b97b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '234344d52dfd4d578fa9afef009c3713', 'request_id': 'req-d1539b9f-4b4f-4ec8-bdc6-55a768a1b97b'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:57.292822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-49ff1db4-6eba-4626-8f16-a66f8e636ce0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd02a1da1edb74420b35ae5a2bd680064', 'request_id': 'req-49ff1db4-6eba-4626-8f16-a66f8e636ce0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:57.308043 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d1539b9f-4b4f-4ec8-bdc6-55a768a1b97b admin admin] The token cache is being invalidate because project 234344d52dfd4d578fa9afef009c3713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:57.309150 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d1539b9f-4b4f-4ec8-bdc6-55a768a1b97b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:57.310237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d1539b9f-4b4f-4ec8-bdc6-55a768a1b97b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:57.316393 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-49ff1db4-6eba-4626-8f16-a66f8e636ce0 admin admin] The token cache is being invalidate because project d02a1da1edb74420b35ae5a2bd680064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:57.316393 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-49ff1db4-6eba-4626-8f16-a66f8e636ce0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:57.317024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 954/1901] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:57 2022] DELETE /identity/v3/projects/234344d52dfd4d578fa9afef009c3713 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:57.317667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-49ff1db4-6eba-4626-8f16-a66f8e636ce0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:57.320801 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 948/1902] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:57 2022] DELETE /identity/v3/projects/d02a1da1edb74420b35ae5a2bd680064 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:57.323591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4cf53-8c5e-461c-85db-1d17a2b1831b None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:57.323879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4cf53-8c5e-461c-85db-1d17a2b1831b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:57.324158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4cf53-8c5e-461c-85db-1d17a2b1831b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:57.325748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912a0c2-ab2e-4282-8df6-cf84b4457135 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:57.326022 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912a0c2-ab2e-4282-8df6-cf84b4457135 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:57.326294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e912a0c2-ab2e-4282-8df6-cf84b4457135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:57.359504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-add4cf53-8c5e-461c-85db-1d17a2b1831b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:57.362478 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-add4cf53-8c5e-461c-85db-1d17a2b1831b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:57.377582 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-e912a0c2-ab2e-4282-8df6-cf84b4457135 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:57.382180 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e912a0c2-ab2e-4282-8df6-cf84b4457135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:57.479477 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 955/1903] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:57.487634 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff48dda-caa4-4023-8e1b-0c5a780036d7 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:57.487791 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff48dda-caa4-4023-8e1b-0c5a780036d7 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:57.488201 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff48dda-caa4-4023-8e1b-0c5a780036d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:57.502899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 949/1904] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:57.507682 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca25b3-2221-4aaa-a094-3d713b56572b None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:57.508215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca25b3-2221-4aaa-a094-3d713b56572b None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:57.508608 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca25b3-2221-4aaa-a094-3d713b56572b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:57.509687 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-0ff48dda-caa4-4023-8e1b-0c5a780036d7 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:57.511029 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-0ff48dda-caa4-4023-8e1b-0c5a780036d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:57.522975 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 956/1905] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:57.523907 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff48dda-caa4-4023-8e1b-0c5a780036d7 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:57.533039 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-09ca25b3-2221-4aaa-a094-3d713b56572b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:57.534143 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff48dda-caa4-4023-8e1b-0c5a780036d7 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:57.534530 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-09ca25b3-2221-4aaa-a094-3d713b56572b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:57.539943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccc563-bb98-48d8-92f7-1e94c39ca126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:57.540768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccc563-bb98-48d8-92f7-1e94c39ca126 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:57.541188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccc563-bb98-48d8-92f7-1e94c39ca126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:57.541597 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccc563-bb98-48d8-92f7-1e94c39ca126 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:57.542389 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccc563-bb98-48d8-92f7-1e94c39ca126 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:57.542923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccc563-bb98-48d8-92f7-1e94c39ca126 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:57.544664 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccc563-bb98-48d8-92f7-1e94c39ca126 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:57.545426 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 950/1906] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:57.549265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca25b3-2221-4aaa-a094-3d713b56572b None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:57.562656 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca25b3-2221-4aaa-a094-3d713b56572b None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:57.563165 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 957/1907] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:58:57.564222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccc563-bb98-48d8-92f7-1e94c39ca126 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:57.570413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a9c75-e6b8-4e21-8e8f-22c789ec8917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:57.571317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a9c75-e6b8-4e21-8e8f-22c789ec8917 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:57.571820 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a9c75-e6b8-4e21-8e8f-22c789ec8917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:57.572124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a9c75-e6b8-4e21-8e8f-22c789ec8917 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:57.573665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a9c75-e6b8-4e21-8e8f-22c789ec8917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:58:57.574054 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a9c75-e6b8-4e21-8e8f-22c789ec8917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:57.575843 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a9c75-e6b8-4e21-8e8f-22c789ec8917 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:57.576937 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccc563-bb98-48d8-92f7-1e94c39ca126 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:57.586407 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871cf40-28fd-4aef-a099-330063b15593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:57.587414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871cf40-28fd-4aef-a099-330063b15593 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:57.588042 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871cf40-28fd-4aef-a099-330063b15593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:57.588680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871cf40-28fd-4aef-a099-330063b15593 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:57.590885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871cf40-28fd-4aef-a099-330063b15593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:57.592004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 951/1908] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:58:57.593397 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871cf40-28fd-4aef-a099-330063b15593 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:57.596251 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a9c75-e6b8-4e21-8e8f-22c789ec8917 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:57.612368 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a9c75-e6b8-4e21-8e8f-22c789ec8917 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:57.618611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768545b6-eea3-4e76-9904-18de216bb278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:57.619432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768545b6-eea3-4e76-9904-18de216bb278 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:57.619952 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768545b6-eea3-4e76-9904-18de216bb278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:57.620737 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768545b6-eea3-4e76-9904-18de216bb278 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:57.621062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768545b6-eea3-4e76-9904-18de216bb278 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:57.623650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768545b6-eea3-4e76-9904-18de216bb278 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:57.631455 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 958/1909] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:57 2022] POST /identity/v3/projects => generated 367 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:57.632150 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871cf40-28fd-4aef-a099-330063b15593 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:57.667183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 952/1910] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:57 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:57.671814 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768545b6-eea3-4e76-9904-18de216bb278 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:57.691450 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:57.699814 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:57.742386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871cf40-28fd-4aef-a099-330063b15593 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:57.795674 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768545b6-eea3-4e76-9904-18de216bb278 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:57.802071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d7fea-e85d-4b84-89c1-a6c0b745395f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:57.802721 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d7fea-e85d-4b84-89c1-a6c0b745395f admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:57.803069 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d7fea-e85d-4b84-89c1-a6c0b745395f admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:57.803285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d7fea-e85d-4b84-89c1-a6c0b745395f admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:57.803919 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d7fea-e85d-4b84-89c1-a6c0b745395f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:57.805819 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d7fea-e85d-4b84-89c1-a6c0b745395f admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:57.843368 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 953/1911] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:57 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:57.847579 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d7fea-e85d-4b84-89c1-a6c0b745395f admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:57.849528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa78ade-0c07-4a65-a23e-99a564891ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:57.850159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa78ade-0c07-4a65-a23e-99a564891ce7 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:57.850434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa78ade-0c07-4a65-a23e-99a564891ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:57.851040 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa78ade-0c07-4a65-a23e-99a564891ce7 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:57.851693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa78ade-0c07-4a65-a23e-99a564891ce7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:57.854352 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa78ade-0c07-4a65-a23e-99a564891ce7 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:57.868929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d7fea-e85d-4b84-89c1-a6c0b745395f admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:57.876666 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b57e2e-79bc-4f36-b326-78bc7d6c24e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:57.877295 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b57e2e-79bc-4f36-b326-78bc7d6c24e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:57.877787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b57e2e-79bc-4f36-b326-78bc7d6c24e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:57.878059 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b57e2e-79bc-4f36-b326-78bc7d6c24e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:57.879100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b57e2e-79bc-4f36-b326-78bc7d6c24e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:57.881699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b57e2e-79bc-4f36-b326-78bc7d6c24e9 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:57.895407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 954/1912] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:57.899771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b57e2e-79bc-4f36-b326-78bc7d6c24e9 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:57.901551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 959/1913] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:57 2022] POST /identity/v3/users => generated 284 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:57.905466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa78ade-0c07-4a65-a23e-99a564891ce7 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:57.918102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b57e2e-79bc-4f36-b326-78bc7d6c24e9 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:57.924306 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc951ae-0b52-4d8e-a834-76dd0558ffec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:57.925003 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc951ae-0b52-4d8e-a834-76dd0558ffec admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:57.925343 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc951ae-0b52-4d8e-a834-76dd0558ffec admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:57.925716 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc951ae-0b52-4d8e-a834-76dd0558ffec admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:57.926414 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc951ae-0b52-4d8e-a834-76dd0558ffec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:57.928649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc951ae-0b52-4d8e-a834-76dd0558ffec admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:57.930509 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa78ade-0c07-4a65-a23e-99a564891ce7 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:57.947494 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 955/1914] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:57.951391 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc951ae-0b52-4d8e-a834-76dd0558ffec admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:57.958856 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a82d1-d069-4dd6-bdbe-2323991d6702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:57.959504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a82d1-d069-4dd6-bdbe-2323991d6702 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:57.959788 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a82d1-d069-4dd6-bdbe-2323991d6702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:57.960061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a82d1-d069-4dd6-bdbe-2323991d6702 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:57.961381 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a82d1-d069-4dd6-bdbe-2323991d6702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:57.962338 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a82d1-d069-4dd6-bdbe-2323991d6702 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:57.966030 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc951ae-0b52-4d8e-a834-76dd0558ffec admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:57.972550 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272087b-541c-481c-acd2-be73fb2b9355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:57.973419 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272087b-541c-481c-acd2-be73fb2b9355 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:57.973760 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272087b-541c-481c-acd2-be73fb2b9355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:57.974519 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272087b-541c-481c-acd2-be73fb2b9355 admin admin] PATH_INFO: `/v3/projects/da5f7d9e8430413d8f48cbd2af089a90/users/f0a8778bfd00425a8810213931d4f071/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:58.008070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272087b-541c-481c-acd2-be73fb2b9355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da5f7d9e8430413d8f48cbd2af089a90, user_id=f0a8778bfd00425a8810213931d4f071, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:58.012440 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272087b-541c-481c-acd2-be73fb2b9355 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:58.019225 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 960/1915] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:58.019836 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a82d1-d069-4dd6-bdbe-2323991d6702 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:58.036804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 956/1916] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:57 2022] PUT /identity/v3/projects/da5f7d9e8430413d8f48cbd2af089a90/users/f0a8778bfd00425a8810213931d4f071/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:58.050222 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad735dd-d6f2-4bd3-ab7c-0833677c9336 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:58.050609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad735dd-d6f2-4bd3-ab7c-0833677c9336 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:58.051081 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad735dd-d6f2-4bd3-ab7c-0833677c9336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:58.072642 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a82d1-d069-4dd6-bdbe-2323991d6702 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:58.085543 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-1ad735dd-d6f2-4bd3-ab7c-0833677c9336 None None] MFA Rules not processed for user `f0a8778bfd00425a8810213931d4f071`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:58.087689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1ad735dd-d6f2-4bd3-ab7c-0833677c9336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:58.246116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 957/1917] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:58.246994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad735dd-d6f2-4bd3-ab7c-0833677c9336 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:58.251904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1ad735dd-d6f2-4bd3-ab7c-0833677c9336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:58.271486 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed3e98-b64c-4bf6-8f9e-200c0f84e3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:58.272183 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed3e98-b64c-4bf6-8f9e-200c0f84e3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:58.272611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed3e98-b64c-4bf6-8f9e-200c0f84e3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:58.273121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed3e98-b64c-4bf6-8f9e-200c0f84e3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:58.274125 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed3e98-b64c-4bf6-8f9e-200c0f84e3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:58.276749 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed3e98-b64c-4bf6-8f9e-200c0f84e3f3 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:58.292245 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 961/1918] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:58.296928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed3e98-b64c-4bf6-8f9e-200c0f84e3f3 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:58.299831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-29ed3e98-b64c-4bf6-8f9e-200c0f84e3f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:58.335472 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:58.367869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad735dd-d6f2-4bd3-ab7c-0833677c9336 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:58.377932 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881247d6-c2ce-4e42-9010-8173ddc19ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:58.378621 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881247d6-c2ce-4e42-9010-8173ddc19ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:58.378959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881247d6-c2ce-4e42-9010-8173ddc19ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:58.379364 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881247d6-c2ce-4e42-9010-8173ddc19ded admin admin] PATH_INFO: `/v3/users/d4f2d6cbec524c908fb0522d80433c48` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:58.381202 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881247d6-c2ce-4e42-9010-8173ddc19ded admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4f2d6cbec524c908fb0522d80433c48)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:58.384844 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881247d6-c2ce-4e42-9010-8173ddc19ded admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:58.397880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed3e98-b64c-4bf6-8f9e-200c0f84e3f3 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:58.406652 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab106f0-97af-49d2-a2cf-47e07813803f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:58.407440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab106f0-97af-49d2-a2cf-47e07813803f service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:58.407794 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab106f0-97af-49d2-a2cf-47e07813803f service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:58.408148 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab106f0-97af-49d2-a2cf-47e07813803f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:58.420336 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab106f0-97af-49d2-a2cf-47e07813803f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:58.426207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab106f0-97af-49d2-a2cf-47e07813803f service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:58.430539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:58.439045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 962/1919] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:58.440728 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab106f0-97af-49d2-a2cf-47e07813803f service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:58.463428 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab106f0-97af-49d2-a2cf-47e07813803f service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:58.470102 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d617030-0655-4bd2-933a-fb55bf804062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:58.471684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d617030-0655-4bd2-933a-fb55bf804062 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:58.472252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d617030-0655-4bd2-933a-fb55bf804062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:58.472671 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d617030-0655-4bd2-933a-fb55bf804062 admin admin] PATH_INFO: `/v3/projects/3b884116df1b4cb9870dbf53a9c5310d/users/761e12c793e04e2d992d06e072f02278/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:58.478955 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-881247d6-c2ce-4e42-9010-8173ddc19ded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4f2d6cbec524c908fb0522d80433c48'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:58.488325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-881247d6-c2ce-4e42-9010-8173ddc19ded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4f2d6cbec524c908fb0522d80433c48'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:58.496713 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d617030-0655-4bd2-933a-fb55bf804062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b884116df1b4cb9870dbf53a9c5310d, user_id=761e12c793e04e2d992d06e072f02278, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:58.501508 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-881247d6-c2ce-4e42-9010-8173ddc19ded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4f2d6cbec524c908fb0522d80433c48'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:58.501807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d617030-0655-4bd2-933a-fb55bf804062 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:58.516983 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-881247d6-c2ce-4e42-9010-8173ddc19ded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4f2d6cbec524c908fb0522d80433c48'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:58.517766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-881247d6-c2ce-4e42-9010-8173ddc19ded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4f2d6cbec524c908fb0522d80433c48'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:58.529729 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 963/1921] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:58 2022] PUT /identity/v3/projects/3b884116df1b4cb9870dbf53a9c5310d/users/761e12c793e04e2d992d06e072f02278/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:58.531795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea647f-425e-4af9-b3b7-cca7ae1ea328 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:58.532083 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea647f-425e-4af9-b3b7-cca7ae1ea328 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:58.532371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea647f-425e-4af9-b3b7-cca7ae1ea328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:58.538213 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 958/1921] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:58 2022] DELETE /identity/v3/users/d4f2d6cbec524c908fb0522d80433c48 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:58.541333 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670811e-689b-4765-84da-31dbb42375bd None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:58.541665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670811e-689b-4765-84da-31dbb42375bd None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:58.541942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670811e-689b-4765-84da-31dbb42375bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:58.556656 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-15ea647f-425e-4af9-b3b7-cca7ae1ea328 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:58.558086 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-15ea647f-425e-4af9-b3b7-cca7ae1ea328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:58.574181 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-3670811e-689b-4765-84da-31dbb42375bd None None] MFA Rules not processed for user `761e12c793e04e2d992d06e072f02278`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:58.575431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3670811e-689b-4765-84da-31dbb42375bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:58.646610 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 964/1922] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:58.655265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea647f-425e-4af9-b3b7-cca7ae1ea328 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:58.659131 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-15ea647f-425e-4af9-b3b7-cca7ae1ea328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:58.682399 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 959/1923] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:58:58.688074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670811e-689b-4765-84da-31dbb42375bd None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:58.689734 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3670811e-689b-4765-84da-31dbb42375bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:58.707197 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670811e-689b-4765-84da-31dbb42375bd None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:58.713156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b93ea-65eb-428b-8623-fad499b362d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:58.713769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b93ea-65eb-428b-8623-fad499b362d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:58.714049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b93ea-65eb-428b-8623-fad499b362d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:58.714322 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b93ea-65eb-428b-8623-fad499b362d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:58.715314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b93ea-65eb-428b-8623-fad499b362d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:58.717171 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b93ea-65eb-428b-8623-fad499b362d6 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:58.754771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 960/1924] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:58:58 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:58:58.757377 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b93ea-65eb-428b-8623-fad499b362d6 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:58.760038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea647f-425e-4af9-b3b7-cca7ae1ea328 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:58.796989 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:58.834096 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23831214-8ac0-45e0-82d5-dd3a2ed230ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:58.834878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23831214-8ac0-45e0-82d5-dd3a2ed230ff service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:58.835223 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23831214-8ac0-45e0-82d5-dd3a2ed230ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:58.835559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23831214-8ac0-45e0-82d5-dd3a2ed230ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:58.837849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b93ea-65eb-428b-8623-fad499b362d6 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:58.843524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d060f-4e6b-488c-b3ed-140727650c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:58.844462 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d060f-4e6b-488c-b3ed-140727650c18 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:58.845187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d060f-4e6b-488c-b3ed-140727650c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:58.845636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d060f-4e6b-488c-b3ed-140727650c18 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:58.847260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d060f-4e6b-488c-b3ed-140727650c18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:58.850050 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d060f-4e6b-488c-b3ed-140727650c18 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:58.851889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23831214-8ac0-45e0-82d5-dd3a2ed230ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:58.853732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23831214-8ac0-45e0-82d5-dd3a2ed230ff service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:58.869607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 965/1925] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:58.879290 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 961/1926] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:58:58 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:58:58.883953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23831214-8ac0-45e0-82d5-dd3a2ed230ff service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:58.897189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23831214-8ac0-45e0-82d5-dd3a2ed230ff service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:58.902396 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325d09a-92a1-4a6a-a489-f1f635f71e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:58.903060 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325d09a-92a1-4a6a-a489-f1f635f71e39 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:58.903338 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325d09a-92a1-4a6a-a489-f1f635f71e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:58.903611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325d09a-92a1-4a6a-a489-f1f635f71e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:58.904406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325d09a-92a1-4a6a-a489-f1f635f71e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:58.912786 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325d09a-92a1-4a6a-a489-f1f635f71e39 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:58.924839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 966/1927] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:58:58.930178 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d060f-4e6b-488c-b3ed-140727650c18 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:58.941162 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d060f-4e6b-488c-b3ed-140727650c18 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:58.946857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32181566-4a73-4c87-8242-72b47ca150d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:58.947512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32181566-4a73-4c87-8242-72b47ca150d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:58.947795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32181566-4a73-4c87-8242-72b47ca150d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:58.948370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32181566-4a73-4c87-8242-72b47ca150d1 admin admin] PATH_INFO: `/v3/projects/b21cb16c87864cd2bb49e616698e135c/users/e5300011aa754e6fa58d94c4293712dc/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:58.964548 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32181566-4a73-4c87-8242-72b47ca150d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21cb16c87864cd2bb49e616698e135c, user_id=e5300011aa754e6fa58d94c4293712dc, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:58.967504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32181566-4a73-4c87-8242-72b47ca150d1 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:58.980963 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 962/1928] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:58 2022] PUT /identity/v3/projects/b21cb16c87864cd2bb49e616698e135c/users/e5300011aa754e6fa58d94c4293712dc/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:58:58.985424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325d09a-92a1-4a6a-a489-f1f635f71e39 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:58.995737 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325d09a-92a1-4a6a-a489-f1f635f71e39 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:59.016076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:59.060988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a588c-8658-4386-a23e-adc86dfc516b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:59.061648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a588c-8658-4386-a23e-adc86dfc516b admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:59.061928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a588c-8658-4386-a23e-adc86dfc516b admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:59.062203 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a588c-8658-4386-a23e-adc86dfc516b admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:59.062887 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a588c-8658-4386-a23e-adc86dfc516b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:59.064482 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a588c-8658-4386-a23e-adc86dfc516b admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:59.078322 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 967/1929] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:58:59.079525 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32181566-4a73-4c87-8242-72b47ca150d1 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:59.091382 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32181566-4a73-4c87-8242-72b47ca150d1 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:59.096532 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abfc79-f400-4d28-8e31-ee8f280a9091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:59.097210 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abfc79-f400-4d28-8e31-ee8f280a9091 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:59.097545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abfc79-f400-4d28-8e31-ee8f280a9091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:59.097824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abfc79-f400-4d28-8e31-ee8f280a9091 admin admin] PATH_INFO: `/v3/projects/b21cb16c87864cd2bb49e616698e135c/users/e5300011aa754e6fa58d94c4293712dc/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:59.099809 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68abfc79-f400-4d28-8e31-ee8f280a9091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21cb16c87864cd2bb49e616698e135c, user_id=e5300011aa754e6fa58d94c4293712dc, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:59.103271 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68abfc79-f400-4d28-8e31-ee8f280a9091 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:59.114457 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 963/1930] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:58:59 2022] PUT /identity/v3/projects/b21cb16c87864cd2bb49e616698e135c/users/e5300011aa754e6fa58d94c4293712dc/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:59.124787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c365d-cfda-46b8-8cff-a0df83f456e7 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:59.125112 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c365d-cfda-46b8-8cff-a0df83f456e7 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:59.125346 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c365d-cfda-46b8-8cff-a0df83f456e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:59.154975 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-fe3c365d-cfda-46b8-8cff-a0df83f456e7 None None] MFA Rules not processed for user `e5300011aa754e6fa58d94c4293712dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:58:59.156347 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-fe3c365d-cfda-46b8-8cff-a0df83f456e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:59.176097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abfc79-f400-4d28-8e31-ee8f280a9091 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:59.177721 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-68abfc79-f400-4d28-8e31-ee8f280a9091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:59.243075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 968/1931] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:58:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:58:59.254927 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c365d-cfda-46b8-8cff-a0df83f456e7 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:59.257302 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-fe3c365d-cfda-46b8-8cff-a0df83f456e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:59.278581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abfc79-f400-4d28-8e31-ee8f280a9091 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:59.283740 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a555f-5238-4d9a-8e57-440f86db5841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:59.284424 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a555f-5238-4d9a-8e57-440f86db5841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:59.284705 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a555f-5238-4d9a-8e57-440f86db5841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:59.284976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a555f-5238-4d9a-8e57-440f86db5841 admin admin] PATH_INFO: `/v3/projects/d68940106e9245d1aefa7c6b64436d61` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:59.286546 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a555f-5238-4d9a-8e57-440f86db5841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d68940106e9245d1aefa7c6b64436d61)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:59.289092 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a555f-5238-4d9a-8e57-440f86db5841 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:59.338669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c365d-cfda-46b8-8cff-a0df83f456e7 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:59.342752 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a087cc8-6f4e-49b3-a7ff-f7e9537f856f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:59.343372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a087cc8-6f4e-49b3-a7ff-f7e9537f856f service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:59.343996 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a087cc8-6f4e-49b3-a7ff-f7e9537f856f service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:59.343996 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a087cc8-6f4e-49b3-a7ff-f7e9537f856f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:59.359359 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a087cc8-6f4e-49b3-a7ff-f7e9537f856f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:59.359686 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-fa8a555f-5238-4d9a-8e57-440f86db5841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd68940106e9245d1aefa7c6b64436d61', 'request_id': 'req-fa8a555f-5238-4d9a-8e57-440f86db5841'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:59.362385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a087cc8-6f4e-49b3-a7ff-f7e9537f856f service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:59.366650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-fa8a555f-5238-4d9a-8e57-440f86db5841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd68940106e9245d1aefa7c6b64436d61', 'request_id': 'req-fa8a555f-5238-4d9a-8e57-440f86db5841'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:59.375321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-fa8a555f-5238-4d9a-8e57-440f86db5841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd68940106e9245d1aefa7c6b64436d61', 'request_id': 'req-fa8a555f-5238-4d9a-8e57-440f86db5841'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:59.375869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 969/1932] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:58:59.393549 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-fa8a555f-5238-4d9a-8e57-440f86db5841 admin admin] The token cache is being invalidate because project d68940106e9245d1aefa7c6b64436d61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:58:59.393887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-fa8a555f-5238-4d9a-8e57-440f86db5841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:59.394544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-fa8a555f-5238-4d9a-8e57-440f86db5841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:59.396470 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 964/1933] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:59 2022] DELETE /identity/v3/projects/d68940106e9245d1aefa7c6b64436d61 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:59.400431 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a087cc8-6f4e-49b3-a7ff-f7e9537f856f service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:59.417143 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a087cc8-6f4e-49b3-a7ff-f7e9537f856f service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:59.479892 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a555f-5238-4d9a-8e57-440f86db5841 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:59.481573 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-fa8a555f-5238-4d9a-8e57-440f86db5841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:59.500616 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:59.505898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:58:59.519048 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579e0e6-747e-4601-adcd-4d179cc7c53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:59.519697 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579e0e6-747e-4601-adcd-4d179cc7c53d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:59.519977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579e0e6-747e-4601-adcd-4d179cc7c53d admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:59.520249 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579e0e6-747e-4601-adcd-4d179cc7c53d admin admin] PATH_INFO: `/v3/users/31bc6032d2d248639b5fe6b518c0041f` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:59.521522 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579e0e6-747e-4601-adcd-4d179cc7c53d admin admin] RBAC: Authorizing `identity:delete_user(user_id=31bc6032d2d248639b5fe6b518c0041f)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:59.523508 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579e0e6-747e-4601-adcd-4d179cc7c53d admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:59.553998 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a555f-5238-4d9a-8e57-440f86db5841 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:58:59.558345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1e02b-9260-42d9-8b39-11e57af83734 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:58:59.559360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1e02b-9260-42d9-8b39-11e57af83734 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:58:59.559641 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1e02b-9260-42d9-8b39-11e57af83734 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:58:59.560103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1e02b-9260-42d9-8b39-11e57af83734 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:58:59.565063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-68e1e02b-9260-42d9-8b39-11e57af83734 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:58:59.574855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e579e0e6-747e-4601-adcd-4d179cc7c53d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31bc6032d2d248639b5fe6b518c0041f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:59.583984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e579e0e6-747e-4601-adcd-4d179cc7c53d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31bc6032d2d248639b5fe6b518c0041f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:59.584602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e579e0e6-747e-4601-adcd-4d179cc7c53d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31bc6032d2d248639b5fe6b518c0041f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:59.598666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e579e0e6-747e-4601-adcd-4d179cc7c53d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31bc6032d2d248639b5fe6b518c0041f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:59.599525 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e579e0e6-747e-4601-adcd-4d179cc7c53d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31bc6032d2d248639b5fe6b518c0041f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:58:59.612246 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 970/1934] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:58:59 2022] DELETE /identity/v3/users/31bc6032d2d248639b5fe6b518c0041f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:58:59.626212 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1e02b-9260-42d9-8b39-11e57af83734 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:58:59.627694 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1e02b-9260-42d9-8b39-11e57af83734 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:58:59.637485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 965/1935] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:58:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:58:59.931306 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579e0e6-747e-4601-adcd-4d179cc7c53d admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:58:59.943953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e579e0e6-747e-4601-adcd-4d179cc7c53d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:00.063936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579e0e6-747e-4601-adcd-4d179cc7c53d admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:00.069052 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1cf46-7187-4518-9c2d-acbb84187f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:00.069694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1cf46-7187-4518-9c2d-acbb84187f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:00.070135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1cf46-7187-4518-9c2d-acbb84187f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:00.070412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1cf46-7187-4518-9c2d-acbb84187f75 admin admin] PATH_INFO: `/v3/projects/755be4b4fa1647a39b4f592331fe8702` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:00.071737 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1cf46-7187-4518-9c2d-acbb84187f75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=755be4b4fa1647a39b4f592331fe8702)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:00.073935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1cf46-7187-4518-9c2d-acbb84187f75 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:00.123997 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c1a1cf46-7187-4518-9c2d-acbb84187f75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '755be4b4fa1647a39b4f592331fe8702', 'request_id': 'req-c1a1cf46-7187-4518-9c2d-acbb84187f75'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.131100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c1a1cf46-7187-4518-9c2d-acbb84187f75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '755be4b4fa1647a39b4f592331fe8702', 'request_id': 'req-c1a1cf46-7187-4518-9c2d-acbb84187f75'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.137443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c1a1cf46-7187-4518-9c2d-acbb84187f75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '755be4b4fa1647a39b4f592331fe8702', 'request_id': 'req-c1a1cf46-7187-4518-9c2d-acbb84187f75'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.142123 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:00.149800 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c1a1cf46-7187-4518-9c2d-acbb84187f75 admin admin] The token cache is being invalidate because project 755be4b4fa1647a39b4f592331fe8702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:00.150306 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c1a1cf46-7187-4518-9c2d-acbb84187f75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.151341 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c1a1cf46-7187-4518-9c2d-acbb84187f75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.152842 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 971/1936] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:58:59 2022] DELETE /identity/v3/projects/755be4b4fa1647a39b4f592331fe8702 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:00.156875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1e02b-9260-42d9-8b39-11e57af83734 service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:00.158211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-68e1e02b-9260-42d9-8b39-11e57af83734 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:00.172495 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1e02b-9260-42d9-8b39-11e57af83734 service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:00.228762 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878de42-8b1a-4630-8b51-4a544be41fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:00.229516 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878de42-8b1a-4630-8b51-4a544be41fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:00.229801 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878de42-8b1a-4630-8b51-4a544be41fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:00.230076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878de42-8b1a-4630-8b51-4a544be41fe7 admin admin] PATH_INFO: `/v3/users/f3481b76a9e34263b219cd0ec53ee8b3` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:00.231219 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878de42-8b1a-4630-8b51-4a544be41fe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3481b76a9e34263b219cd0ec53ee8b3)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:00.233130 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878de42-8b1a-4630-8b51-4a544be41fe7 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:00.278503 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a878de42-8b1a-4630-8b51-4a544be41fe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3481b76a9e34263b219cd0ec53ee8b3'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.285776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a878de42-8b1a-4630-8b51-4a544be41fe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3481b76a9e34263b219cd0ec53ee8b3'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.293414 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a878de42-8b1a-4630-8b51-4a544be41fe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3481b76a9e34263b219cd0ec53ee8b3'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.298620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:00.307038 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a878de42-8b1a-4630-8b51-4a544be41fe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3481b76a9e34263b219cd0ec53ee8b3'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.307724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a878de42-8b1a-4630-8b51-4a544be41fe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3481b76a9e34263b219cd0ec53ee8b3'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.313920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 966/1937] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:00 2022] DELETE /identity/v3/users/f3481b76a9e34263b219cd0ec53ee8b3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:00.724372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1cf46-7187-4518-9c2d-acbb84187f75 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:00.725882 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-c1a1cf46-7187-4518-9c2d-acbb84187f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:00.738702 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1cf46-7187-4518-9c2d-acbb84187f75 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:00.810174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbd2cd-3e00-44f3-9638-712f306305f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:00.810849 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbd2cd-3e00-44f3-9638-712f306305f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:00.811124 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbd2cd-3e00-44f3-9638-712f306305f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:00.811397 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbd2cd-3e00-44f3-9638-712f306305f1 admin admin] PATH_INFO: `/v3/projects/50d7421b77504bf4a43ab6e1dc9f203e` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:00.812514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbd2cd-3e00-44f3-9638-712f306305f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50d7421b77504bf4a43ab6e1dc9f203e)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:00.814441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbd2cd-3e00-44f3-9638-712f306305f1 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:00.860768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-96cbd2cd-3e00-44f3-9638-712f306305f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50d7421b77504bf4a43ab6e1dc9f203e', 'request_id': 'req-96cbd2cd-3e00-44f3-9638-712f306305f1'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.868192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-96cbd2cd-3e00-44f3-9638-712f306305f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50d7421b77504bf4a43ab6e1dc9f203e', 'request_id': 'req-96cbd2cd-3e00-44f3-9638-712f306305f1'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.874724 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-96cbd2cd-3e00-44f3-9638-712f306305f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50d7421b77504bf4a43ab6e1dc9f203e', 'request_id': 'req-96cbd2cd-3e00-44f3-9638-712f306305f1'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.885802 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-96cbd2cd-3e00-44f3-9638-712f306305f1 admin admin] The token cache is being invalidate because project 50d7421b77504bf4a43ab6e1dc9f203e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:00.886085 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-96cbd2cd-3e00-44f3-9638-712f306305f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.886705 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-96cbd2cd-3e00-44f3-9638-712f306305f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:00.888207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 972/1938] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:00 2022] DELETE /identity/v3/projects/50d7421b77504bf4a43ab6e1dc9f203e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:00.894197 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe6cf2-ecc8-4be5-a835-7e2094c46aef None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:00.894414 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe6cf2-ecc8-4be5-a835-7e2094c46aef None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:00.894683 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe6cf2-ecc8-4be5-a835-7e2094c46aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:00.914060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-74fe6cf2-ecc8-4be5-a835-7e2094c46aef None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:00.915284 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-74fe6cf2-ecc8-4be5-a835-7e2094c46aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:00.974698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 967/1939] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:00.984406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61a7a4-b3e7-4595-b880-3d1d1fbe9175 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:00.984683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61a7a4-b3e7-4595-b880-3d1d1fbe9175 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:00.984955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61a7a4-b3e7-4595-b880-3d1d1fbe9175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.006373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-1e61a7a4-b3e7-4595-b880-3d1d1fbe9175 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:01.007736 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-1e61a7a4-b3e7-4595-b880-3d1d1fbe9175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:01.023976 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 973/1940] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:01.028245 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe6cf2-ecc8-4be5-a835-7e2094c46aef None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.040401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe6cf2-ecc8-4be5-a835-7e2094c46aef None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.045692 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbc560-2140-4f5a-b193-547d40f8a9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.046546 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbc560-2140-4f5a-b193-547d40f8a9d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.046981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbc560-2140-4f5a-b193-547d40f8a9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.047338 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbc560-2140-4f5a-b193-547d40f8a9d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.048049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbc560-2140-4f5a-b193-547d40f8a9d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:01.048416 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbc560-2140-4f5a-b193-547d40f8a9d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:01.050221 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbc560-2140-4f5a-b193-547d40f8a9d9 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:01.069873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 968/1941] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:59:01.070854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61a7a4-b3e7-4595-b880-3d1d1fbe9175 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.082475 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61a7a4-b3e7-4595-b880-3d1d1fbe9175 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.094964 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02398b-03f4-45a0-aa86-da1af3364d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.095778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02398b-03f4-45a0-aa86-da1af3364d6f admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.096639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02398b-03f4-45a0-aa86-da1af3364d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.096639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02398b-03f4-45a0-aa86-da1af3364d6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.097081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02398b-03f4-45a0-aa86-da1af3364d6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:01.099141 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02398b-03f4-45a0-aa86-da1af3364d6f admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:01.130792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 974/1942] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:01 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:01.134185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbc560-2140-4f5a-b193-547d40f8a9d9 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.152519 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbc560-2140-4f5a-b193-547d40f8a9d9 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.250723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737c444-6d4f-4e98-bf60-0fc3ce3784af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.251494 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737c444-6d4f-4e98-bf60-0fc3ce3784af admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.251846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737c444-6d4f-4e98-bf60-0fc3ce3784af admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.252177 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737c444-6d4f-4e98-bf60-0fc3ce3784af admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.252879 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737c444-6d4f-4e98-bf60-0fc3ce3784af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:01.254904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737c444-6d4f-4e98-bf60-0fc3ce3784af admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:01.291655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 969/1943] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:01 2022] POST /identity/v3/users => generated 290 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:01.295678 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02398b-03f4-45a0-aa86-da1af3364d6f admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.308987 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02398b-03f4-45a0-aa86-da1af3364d6f admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.314323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98b9bb-eaf2-4409-bae1-52d846bf43df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.315029 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98b9bb-eaf2-4409-bae1-52d846bf43df admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.315308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98b9bb-eaf2-4409-bae1-52d846bf43df admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.315577 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98b9bb-eaf2-4409-bae1-52d846bf43df admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.316223 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98b9bb-eaf2-4409-bae1-52d846bf43df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:01.318098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98b9bb-eaf2-4409-bae1-52d846bf43df admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:01.334051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 975/1944] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:01.335261 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737c444-6d4f-4e98-bf60-0fc3ce3784af admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.364073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737c444-6d4f-4e98-bf60-0fc3ce3784af admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.371416 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b72e89-c09c-4cc9-b203-a9f65abc4226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.372582 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b72e89-c09c-4cc9-b203-a9f65abc4226 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.372582 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b72e89-c09c-4cc9-b203-a9f65abc4226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.373135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b72e89-c09c-4cc9-b203-a9f65abc4226 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.373964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b72e89-c09c-4cc9-b203-a9f65abc4226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:01.376304 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b72e89-c09c-4cc9-b203-a9f65abc4226 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:01.392278 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 970/1945] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:01.396173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98b9bb-eaf2-4409-bae1-52d846bf43df admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.408367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98b9bb-eaf2-4409-bae1-52d846bf43df admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.415209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f200da-9901-4f3e-887c-5d96eb834df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.416207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f200da-9901-4f3e-887c-5d96eb834df7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.416583 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f200da-9901-4f3e-887c-5d96eb834df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.416950 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f200da-9901-4f3e-887c-5d96eb834df7 admin admin] PATH_INFO: `/v3/projects/23f8e1537a634f2f9cda7b9b4df0c135/users/94a910c81fa24e5e88ffda6cc9b86100/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.433672 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f200da-9901-4f3e-887c-5d96eb834df7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23f8e1537a634f2f9cda7b9b4df0c135, user_id=94a910c81fa24e5e88ffda6cc9b86100, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:01.436591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f200da-9901-4f3e-887c-5d96eb834df7 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:01.452937 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 976/1946] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:01 2022] PUT /identity/v3/projects/23f8e1537a634f2f9cda7b9b4df0c135/users/94a910c81fa24e5e88ffda6cc9b86100/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:01.464297 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97917e07-33c2-49c1-ab65-a24c2bd49988 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.464942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97917e07-33c2-49c1-ab65-a24c2bd49988 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.465292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97917e07-33c2-49c1-ab65-a24c2bd49988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.499072 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-97917e07-33c2-49c1-ab65-a24c2bd49988 None None] MFA Rules not processed for user `94a910c81fa24e5e88ffda6cc9b86100`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:01.500387 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-97917e07-33c2-49c1-ab65-a24c2bd49988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:01.557158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:01.595316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 971/1947] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:01.600219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f200da-9901-4f3e-887c-5d96eb834df7 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.633907 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:01.690537 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f200da-9901-4f3e-887c-5d96eb834df7 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.697027 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dd831-fb61-41bb-9db4-4b033a534332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.697996 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dd831-fb61-41bb-9db4-4b033a534332 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.698457 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dd831-fb61-41bb-9db4-4b033a534332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.698838 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dd831-fb61-41bb-9db4-4b033a534332 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.699587 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489dd831-fb61-41bb-9db4-4b033a534332 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:01.701638 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489dd831-fb61-41bb-9db4-4b033a534332 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:01.757986 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 977/1948] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:01 2022] POST /identity/v3/projects => generated 379 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:01.762551 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97917e07-33c2-49c1-ab65-a24c2bd49988 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.776174 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97917e07-33c2-49c1-ab65-a24c2bd49988 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.853442 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43d966-e0cb-4ab5-a7f9-bad38c245018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.854647 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43d966-e0cb-4ab5-a7f9-bad38c245018 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.855186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43d966-e0cb-4ab5-a7f9-bad38c245018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.855854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43d966-e0cb-4ab5-a7f9-bad38c245018 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.857360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43d966-e0cb-4ab5-a7f9-bad38c245018 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:01.859607 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43d966-e0cb-4ab5-a7f9-bad38c245018 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:01.887914 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 972/1949] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:01 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:01.892040 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dd831-fb61-41bb-9db4-4b033a534332 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.907441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dd831-fb61-41bb-9db4-4b033a534332 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.915502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dcb175-c517-4797-bc1b-044342a6abf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.916267 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcb175-c517-4797-bc1b-044342a6abf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.916671 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcb175-c517-4797-bc1b-044342a6abf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.917041 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcb175-c517-4797-bc1b-044342a6abf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.917921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dcb175-c517-4797-bc1b-044342a6abf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:01.919810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dcb175-c517-4797-bc1b-044342a6abf8 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:01.931265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 978/1950] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:01.935473 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43d966-e0cb-4ab5-a7f9-bad38c245018 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.947928 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43d966-e0cb-4ab5-a7f9-bad38c245018 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.952895 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d9b8e-2754-48c0-8fef-05368a5b532d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.953495 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d9b8e-2754-48c0-8fef-05368a5b532d admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.953768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d9b8e-2754-48c0-8fef-05368a5b532d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.954040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d9b8e-2754-48c0-8fef-05368a5b532d admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:01.954688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d9b8e-2754-48c0-8fef-05368a5b532d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:01.956479 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d9b8e-2754-48c0-8fef-05368a5b532d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:01.970995 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 973/1951] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:01.974279 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dcb175-c517-4797-bc1b-044342a6abf8 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:01.988086 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dcb175-c517-4797-bc1b-044342a6abf8 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:01.993458 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ac388-8bb9-44eb-88f7-cd98b470f732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:01.994109 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ac388-8bb9-44eb-88f7-cd98b470f732 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:01.994410 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ac388-8bb9-44eb-88f7-cd98b470f732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:01.994674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ac388-8bb9-44eb-88f7-cd98b470f732 admin admin] PATH_INFO: `/v3/projects/141b2d7b54714739b04d34677fc19494/users/6d895fbe4ee146f9afbfcdc1af27af9a/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:02.015535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ac388-8bb9-44eb-88f7-cd98b470f732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=141b2d7b54714739b04d34677fc19494, user_id=6d895fbe4ee146f9afbfcdc1af27af9a, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:02.019516 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ac388-8bb9-44eb-88f7-cd98b470f732 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:02.047422 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 979/1952] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:01 2022] PUT /identity/v3/projects/141b2d7b54714739b04d34677fc19494/users/6d895fbe4ee146f9afbfcdc1af27af9a/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:02.061887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebb72c-401d-45b2-9794-8d258dc017c7 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:02.062274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebb72c-401d-45b2-9794-8d258dc017c7 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:02.062674 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebb72c-401d-45b2-9794-8d258dc017c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:02.101650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-22ebb72c-401d-45b2-9794-8d258dc017c7 None None] MFA Rules not processed for user `6d895fbe4ee146f9afbfcdc1af27af9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:02.104099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-22ebb72c-401d-45b2-9794-8d258dc017c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:02.266193 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 974/1953] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:02.271119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ac388-8bb9-44eb-88f7-cd98b470f732 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:02.373425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ac388-8bb9-44eb-88f7-cd98b470f732 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:02.378761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7a6d1-a881-4d1c-a5c3-8c1919687381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:02.379352 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7a6d1-a881-4d1c-a5c3-8c1919687381 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:02.379695 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7a6d1-a881-4d1c-a5c3-8c1919687381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:02.379930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7a6d1-a881-4d1c-a5c3-8c1919687381 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:02.380914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7a6d1-a881-4d1c-a5c3-8c1919687381 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:02.383084 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7a6d1-a881-4d1c-a5c3-8c1919687381 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:02.448529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebb72c-401d-45b2-9794-8d258dc017c7 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:02.449442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 980/1954] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:02 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:02.471404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebb72c-401d-45b2-9794-8d258dc017c7 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:02.548284 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:02.563807 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55879bb9-57ed-471c-82bb-11d52c59cfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:02.564283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55879bb9-57ed-471c-82bb-11d52c59cfbe admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:02.564632 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55879bb9-57ed-471c-82bb-11d52c59cfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:02.564972 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55879bb9-57ed-471c-82bb-11d52c59cfbe admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:02.565741 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55879bb9-57ed-471c-82bb-11d52c59cfbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:02.567948 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55879bb9-57ed-471c-82bb-11d52c59cfbe admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:02.592730 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 975/1955] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:02 2022] POST /identity/v3/users => generated 295 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:02.596878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7a6d1-a881-4d1c-a5c3-8c1919687381 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:02.616057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7a6d1-a881-4d1c-a5c3-8c1919687381 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:02.621092 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23f962-e513-49e2-8456-1d3833d2c2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:02.621715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23f962-e513-49e2-8456-1d3833d2c2cc admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:02.622031 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23f962-e513-49e2-8456-1d3833d2c2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:02.622328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23f962-e513-49e2-8456-1d3833d2c2cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:02.623000 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23f962-e513-49e2-8456-1d3833d2c2cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:02.624854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23f962-e513-49e2-8456-1d3833d2c2cc admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:02.638815 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 981/1956] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:02.641497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55879bb9-57ed-471c-82bb-11d52c59cfbe admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:02.652171 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55879bb9-57ed-471c-82bb-11d52c59cfbe admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:02.656810 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3e137-b6cf-4ec3-8725-25e469f8b362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:02.657465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3e137-b6cf-4ec3-8725-25e469f8b362 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:02.657745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3e137-b6cf-4ec3-8725-25e469f8b362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:02.658019 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3e137-b6cf-4ec3-8725-25e469f8b362 admin admin] PATH_INFO: `/v3/projects/ae0353ce58a240689a6b365b391cb550/users/93e17e326ecd44539577014dc89e2083/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:02.675705 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3e137-b6cf-4ec3-8725-25e469f8b362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae0353ce58a240689a6b365b391cb550, user_id=93e17e326ecd44539577014dc89e2083, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:02.678375 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3e137-b6cf-4ec3-8725-25e469f8b362 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:02.691902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 976/1957] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:02 2022] PUT /identity/v3/projects/ae0353ce58a240689a6b365b391cb550/users/93e17e326ecd44539577014dc89e2083/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:02.696363 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23f962-e513-49e2-8456-1d3833d2c2cc admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:02.706222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23f962-e513-49e2-8456-1d3833d2c2cc admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:02.750109 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:02.764967 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0245e5e-7d5b-4487-8bf8-ff6729d73e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:02.765592 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0245e5e-7d5b-4487-8bf8-ff6729d73e26 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:02.765868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0245e5e-7d5b-4487-8bf8-ff6729d73e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:02.766136 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0245e5e-7d5b-4487-8bf8-ff6729d73e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:02.766826 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0245e5e-7d5b-4487-8bf8-ff6729d73e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:02.768418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0245e5e-7d5b-4487-8bf8-ff6729d73e26 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:02.778879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 982/1958] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:02.783262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3e137-b6cf-4ec3-8725-25e469f8b362 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:02.793503 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3e137-b6cf-4ec3-8725-25e469f8b362 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:02.798101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698b438-5313-432e-8a53-2fd07052cb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:02.798754 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698b438-5313-432e-8a53-2fd07052cb41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:02.799082 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698b438-5313-432e-8a53-2fd07052cb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:02.799349 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698b438-5313-432e-8a53-2fd07052cb41 admin admin] PATH_INFO: `/v3/projects/ae0353ce58a240689a6b365b391cb550/users/93e17e326ecd44539577014dc89e2083/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:02.801273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698b438-5313-432e-8a53-2fd07052cb41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae0353ce58a240689a6b365b391cb550, user_id=93e17e326ecd44539577014dc89e2083, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:02.803968 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698b438-5313-432e-8a53-2fd07052cb41 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:02.814079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 977/1959] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:02 2022] PUT /identity/v3/projects/ae0353ce58a240689a6b365b391cb550/users/93e17e326ecd44539577014dc89e2083/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:02.824835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb71114-f979-4316-bdbe-04ad532ea8f4 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:02.825124 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb71114-f979-4316-bdbe-04ad532ea8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:02.825402 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb71114-f979-4316-bdbe-04ad532ea8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:02.853566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-ffb71114-f979-4316-bdbe-04ad532ea8f4 None None] MFA Rules not processed for user `93e17e326ecd44539577014dc89e2083`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:02.854826 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ffb71114-f979-4316-bdbe-04ad532ea8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:02.925375 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 983/1960] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:02.937578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698b438-5313-432e-8a53-2fd07052cb41 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:02.939252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a698b438-5313-432e-8a53-2fd07052cb41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:02.996095 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698b438-5313-432e-8a53-2fd07052cb41 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:02.999700 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382750f-a58d-4d74-abbc-1636f61aae40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:03.000298 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382750f-a58d-4d74-abbc-1636f61aae40 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:03.000573 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382750f-a58d-4d74-abbc-1636f61aae40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:03.000840 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382750f-a58d-4d74-abbc-1636f61aae40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:03.010464 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e382750f-a58d-4d74-abbc-1636f61aae40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:03.011926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e382750f-a58d-4d74-abbc-1636f61aae40 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:03.021838 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 978/1961] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:03.064157 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb71114-f979-4316-bdbe-04ad532ea8f4 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:03.065652 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ffb71114-f979-4316-bdbe-04ad532ea8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:03.079028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb71114-f979-4316-bdbe-04ad532ea8f4 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:03.084038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566699aa-a8cb-45af-b8fe-c1e380ac39e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:03.084740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566699aa-a8cb-45af-b8fe-c1e380ac39e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:03.085056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566699aa-a8cb-45af-b8fe-c1e380ac39e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:03.085718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566699aa-a8cb-45af-b8fe-c1e380ac39e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:03.145826 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566699aa-a8cb-45af-b8fe-c1e380ac39e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:03.147292 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566699aa-a8cb-45af-b8fe-c1e380ac39e9 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:03.156341 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 984/1962] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:06.361025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382750f-a58d-4d74-abbc-1636f61aae40 service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:06.370312 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566699aa-a8cb-45af-b8fe-c1e380ac39e9 service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:06.371829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e382750f-a58d-4d74-abbc-1636f61aae40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:06.371941 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-566699aa-a8cb-45af-b8fe-c1e380ac39e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:06.465580 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382750f-a58d-4d74-abbc-1636f61aae40 service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:06.472029 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566699aa-a8cb-45af-b8fe-c1e380ac39e9 service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:06.473908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f59b40-0115-4b9e-b208-6af972f1661f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:06.474629 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f59b40-0115-4b9e-b208-6af972f1661f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:06.475012 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f59b40-0115-4b9e-b208-6af972f1661f admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:06.475248 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f59b40-0115-4b9e-b208-6af972f1661f admin admin] PATH_INFO: `/v3/users/761e12c793e04e2d992d06e072f02278` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:06.476527 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f59b40-0115-4b9e-b208-6af972f1661f admin admin] RBAC: Authorizing `identity:delete_user(user_id=761e12c793e04e2d992d06e072f02278)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:06.479003 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f59b40-0115-4b9e-b208-6af972f1661f admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:06.479157 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b9def-cb8d-4a16-a865-85ab500daa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:06.480265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b9def-cb8d-4a16-a865-85ab500daa38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:06.480619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b9def-cb8d-4a16-a865-85ab500daa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:06.482932 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b9def-cb8d-4a16-a865-85ab500daa38 admin admin] PATH_INFO: `/v3/users/2f06cb08ff8c4100b82f8566d2e28927` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:06.485055 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b9def-cb8d-4a16-a865-85ab500daa38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f06cb08ff8c4100b82f8566d2e28927)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:06.487318 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b9def-cb8d-4a16-a865-85ab500daa38 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:06.495067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:06.557524 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-141b9def-cb8d-4a16-a865-85ab500daa38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f06cb08ff8c4100b82f8566d2e28927'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:06.561043 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-63f59b40-0115-4b9e-b208-6af972f1661f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '761e12c793e04e2d992d06e072f02278'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:06.563780 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:06.567610 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-141b9def-cb8d-4a16-a865-85ab500daa38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f06cb08ff8c4100b82f8566d2e28927'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:06.567908 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-141b9def-cb8d-4a16-a865-85ab500daa38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f06cb08ff8c4100b82f8566d2e28927'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:06.575702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-63f59b40-0115-4b9e-b208-6af972f1661f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '761e12c793e04e2d992d06e072f02278'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:06.584329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-63f59b40-0115-4b9e-b208-6af972f1661f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '761e12c793e04e2d992d06e072f02278'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:06.598162 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-141b9def-cb8d-4a16-a865-85ab500daa38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f06cb08ff8c4100b82f8566d2e28927'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:06.599175 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-141b9def-cb8d-4a16-a865-85ab500daa38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f06cb08ff8c4100b82f8566d2e28927'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:06.601804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-63f59b40-0115-4b9e-b208-6af972f1661f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '761e12c793e04e2d992d06e072f02278'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:06.603327 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-63f59b40-0115-4b9e-b208-6af972f1661f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '761e12c793e04e2d992d06e072f02278'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:06.613379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 979/1963] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:06 2022] DELETE /identity/v3/users/761e12c793e04e2d992d06e072f02278 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:06.616723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 985/1964] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:06 2022] DELETE /identity/v3/users/2f06cb08ff8c4100b82f8566d2e28927 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:06.623745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826c020-426f-47c6-99b9-e3da6b3ef769 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:06.624226 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826c020-426f-47c6-99b9-e3da6b3ef769 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:06.624679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826c020-426f-47c6-99b9-e3da6b3ef769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:06.629233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5225a-8b60-480a-8e22-4b52c788e771 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:06.629713 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5225a-8b60-480a-8e22-4b52c788e771 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:06.630323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5225a-8b60-480a-8e22-4b52c788e771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:06.653762 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-9826c020-426f-47c6-99b9-e3da6b3ef769 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:06.654562 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-c0f5225a-8b60-480a-8e22-4b52c788e771 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:06.655145 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-9826c020-426f-47c6-99b9-e3da6b3ef769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:06.655833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-c0f5225a-8b60-480a-8e22-4b52c788e771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:06.721728 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 980/1965] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:06.724186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 986/1966] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:06.733016 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826c020-426f-47c6-99b9-e3da6b3ef769 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:06.734277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5225a-8b60-480a-8e22-4b52c788e771 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:06.734624 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-9826c020-426f-47c6-99b9-e3da6b3ef769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:06.735850 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-c0f5225a-8b60-480a-8e22-4b52c788e771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:06.807118 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5225a-8b60-480a-8e22-4b52c788e771 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:06.807440 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826c020-426f-47c6-99b9-e3da6b3ef769 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:06.811068 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4784e4a-7e3e-42fb-867b-2358cedf43b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:06.811200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ab500-df9e-4f7e-9444-ca7dd6ecfe7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:06.811622 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4784e4a-7e3e-42fb-867b-2358cedf43b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:06.811730 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ab500-df9e-4f7e-9444-ca7dd6ecfe7e service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:06.811906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4784e4a-7e3e-42fb-867b-2358cedf43b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:06.812008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ab500-df9e-4f7e-9444-ca7dd6ecfe7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:06.812177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4784e4a-7e3e-42fb-867b-2358cedf43b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:06.812278 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ab500-df9e-4f7e-9444-ca7dd6ecfe7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:06.821705 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ab500-df9e-4f7e-9444-ca7dd6ecfe7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:06.822105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4784e4a-7e3e-42fb-867b-2358cedf43b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:06.823205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ab500-df9e-4f7e-9444-ca7dd6ecfe7e service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:06.823587 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4784e4a-7e3e-42fb-867b-2358cedf43b8 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:06.832966 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 981/1967] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:59:06.833124 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 987/1968] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:59:07.127817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4784e4a-7e3e-42fb-867b-2358cedf43b8 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:07.131206 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ab500-df9e-4f7e-9444-ca7dd6ecfe7e service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:07.133163 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b4784e4a-7e3e-42fb-867b-2358cedf43b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:07.134084 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-085ab500-df9e-4f7e-9444-ca7dd6ecfe7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:07.158820 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ab500-df9e-4f7e-9444-ca7dd6ecfe7e service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:07.162145 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4784e4a-7e3e-42fb-867b-2358cedf43b8 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:07.166188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db6f3bc-7071-43b0-b15a-541c5b861fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:07.166959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6f3bc-7071-43b0-b15a-541c5b861fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:07.167125 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6f3bc-7071-43b0-b15a-541c5b861fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:07.167219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:07.167403 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db6f3bc-7071-43b0-b15a-541c5b861fdb admin admin] PATH_INFO: `/v3/projects/3b884116df1b4cb9870dbf53a9c5310d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:07.167844 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:07.168124 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:07.168447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799 admin admin] PATH_INFO: `/v3/projects/80320cebc6624a18b54bb683d7dabab0` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:07.168651 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db6f3bc-7071-43b0-b15a-541c5b861fdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b884116df1b4cb9870dbf53a9c5310d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:07.169514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80320cebc6624a18b54bb683d7dabab0)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:07.171002 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db6f3bc-7071-43b0-b15a-541c5b861fdb admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:07.171476 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:07.200866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:07.225638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8db6f3bc-7071-43b0-b15a-541c5b861fdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b884116df1b4cb9870dbf53a9c5310d', 'request_id': 'req-8db6f3bc-7071-43b0-b15a-541c5b861fdb'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.227241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80320cebc6624a18b54bb683d7dabab0', 'request_id': 'req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.232518 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8db6f3bc-7071-43b0-b15a-541c5b861fdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b884116df1b4cb9870dbf53a9c5310d', 'request_id': 'req-8db6f3bc-7071-43b0-b15a-541c5b861fdb'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.236268 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80320cebc6624a18b54bb683d7dabab0', 'request_id': 'req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.240160 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8db6f3bc-7071-43b0-b15a-541c5b861fdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b884116df1b4cb9870dbf53a9c5310d', 'request_id': 'req-8db6f3bc-7071-43b0-b15a-541c5b861fdb'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.241862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:INSERT=1,keystone:DELETE=9,keystone:SELECT=90 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:07.243163 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80320cebc6624a18b54bb683d7dabab0', 'request_id': 'req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.255380 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8db6f3bc-7071-43b0-b15a-541c5b861fdb admin admin] The token cache is being invalidate because project 3b884116df1b4cb9870dbf53a9c5310d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:07.255671 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8db6f3bc-7071-43b0-b15a-541c5b861fdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.255973 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799 admin admin] The token cache is being invalidate because project 80320cebc6624a18b54bb683d7dabab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:07.256267 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.256372 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8db6f3bc-7071-43b0-b15a-541c5b861fdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.256913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.258138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 982/1969] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:07 2022] DELETE /identity/v3/projects/3b884116df1b4cb9870dbf53a9c5310d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:07.258396 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 988/1970] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:07 2022] DELETE /identity/v3/projects/80320cebc6624a18b54bb683d7dabab0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:07.262243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db6f3bc-7071-43b0-b15a-541c5b861fdb admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:07.263032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:07.263602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-8db6f3bc-7071-43b0-b15a-541c5b861fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:07.264368 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:07.334079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db6f3bc-7071-43b0-b15a-541c5b861fdb admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:07.335070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb150b-8ec3-49cd-a3cb-f1fd613ee799 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:07.339926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d02e1a-aeaf-4965-86b9-e039bb22fbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:07.340587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d02e1a-aeaf-4965-86b9-e039bb22fbe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:07.340865 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d02e1a-aeaf-4965-86b9-e039bb22fbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:07.340964 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab91dc0-05cb-483b-a4cf-68b8ce10db83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:07.341142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d02e1a-aeaf-4965-86b9-e039bb22fbe4 admin admin] PATH_INFO: `/v3/users/e5300011aa754e6fa58d94c4293712dc` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:07.341505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab91dc0-05cb-483b-a4cf-68b8ce10db83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:07.341787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab91dc0-05cb-483b-a4cf-68b8ce10db83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:07.342064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab91dc0-05cb-483b-a4cf-68b8ce10db83 admin admin] PATH_INFO: `/v3/users/aa8cd9e2d8c447d5a598f98cacd536eb` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:07.342415 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d02e1a-aeaf-4965-86b9-e039bb22fbe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5300011aa754e6fa58d94c4293712dc)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:07.343232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab91dc0-05cb-483b-a4cf-68b8ce10db83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa8cd9e2d8c447d5a598f98cacd536eb)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:07.344394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d02e1a-aeaf-4965-86b9-e039bb22fbe4 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:07.345138 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab91dc0-05cb-483b-a4cf-68b8ce10db83 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:07.416158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5ab91dc0-05cb-483b-a4cf-68b8ce10db83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa8cd9e2d8c447d5a598f98cacd536eb'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.434636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5ab91dc0-05cb-483b-a4cf-68b8ce10db83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa8cd9e2d8c447d5a598f98cacd536eb'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.436149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-18d02e1a-aeaf-4965-86b9-e039bb22fbe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5300011aa754e6fa58d94c4293712dc'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.439237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5ab91dc0-05cb-483b-a4cf-68b8ce10db83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa8cd9e2d8c447d5a598f98cacd536eb'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.458011 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-18d02e1a-aeaf-4965-86b9-e039bb22fbe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5300011aa754e6fa58d94c4293712dc'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.472171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5ab91dc0-05cb-483b-a4cf-68b8ce10db83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa8cd9e2d8c447d5a598f98cacd536eb'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.473351 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5ab91dc0-05cb-483b-a4cf-68b8ce10db83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa8cd9e2d8c447d5a598f98cacd536eb'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.477853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-18d02e1a-aeaf-4965-86b9-e039bb22fbe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5300011aa754e6fa58d94c4293712dc'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.498100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 989/1971] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:07 2022] DELETE /identity/v3/users/aa8cd9e2d8c447d5a598f98cacd536eb => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:07.507090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-18d02e1a-aeaf-4965-86b9-e039bb22fbe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5300011aa754e6fa58d94c4293712dc'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.508545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-18d02e1a-aeaf-4965-86b9-e039bb22fbe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5300011aa754e6fa58d94c4293712dc'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:07.527037 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 983/1972] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:07 2022] DELETE /identity/v3/users/e5300011aa754e6fa58d94c4293712dc => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:08.076092 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab91dc0-05cb-483b-a4cf-68b8ce10db83 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:08.082486 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-5ab91dc0-05cb-483b-a4cf-68b8ce10db83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:08.093369 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d02e1a-aeaf-4965-86b9-e039bb22fbe4 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:08.095272 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-18d02e1a-aeaf-4965-86b9-e039bb22fbe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:08.124615 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:08.160116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab91dc0-05cb-483b-a4cf-68b8ce10db83 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:08.188740 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d02e1a-aeaf-4965-86b9-e039bb22fbe4 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:08.194220 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937c00c-a0af-4098-9f16-1f6a242380dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:08.194891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937c00c-a0af-4098-9f16-1f6a242380dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.195171 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937c00c-a0af-4098-9f16-1f6a242380dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.195463 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937c00c-a0af-4098-9f16-1f6a242380dd admin admin] PATH_INFO: `/v3/projects/b21cb16c87864cd2bb49e616698e135c` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.196586 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937c00c-a0af-4098-9f16-1f6a242380dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21cb16c87864cd2bb49e616698e135c)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:08.198571 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937c00c-a0af-4098-9f16-1f6a242380dd admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:08.210180 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:08.218758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63879ac-48f3-47ce-be3d-01b1b2680eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:08.219413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63879ac-48f3-47ce-be3d-01b1b2680eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.219689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63879ac-48f3-47ce-be3d-01b1b2680eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.219964 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63879ac-48f3-47ce-be3d-01b1b2680eb6 admin admin] PATH_INFO: `/v3/projects/1f021e1ce06a41a494fa261156b871e3` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.221099 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63879ac-48f3-47ce-be3d-01b1b2680eb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f021e1ce06a41a494fa261156b871e3)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:08.223024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63879ac-48f3-47ce-be3d-01b1b2680eb6 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:08.250098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-5937c00c-a0af-4098-9f16-1f6a242380dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21cb16c87864cd2bb49e616698e135c', 'request_id': 'req-5937c00c-a0af-4098-9f16-1f6a242380dd'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:08.257737 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-5937c00c-a0af-4098-9f16-1f6a242380dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21cb16c87864cd2bb49e616698e135c', 'request_id': 'req-5937c00c-a0af-4098-9f16-1f6a242380dd'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:08.263533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-5937c00c-a0af-4098-9f16-1f6a242380dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21cb16c87864cd2bb49e616698e135c', 'request_id': 'req-5937c00c-a0af-4098-9f16-1f6a242380dd'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:08.270683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d63879ac-48f3-47ce-be3d-01b1b2680eb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f021e1ce06a41a494fa261156b871e3', 'request_id': 'req-d63879ac-48f3-47ce-be3d-01b1b2680eb6'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:08.278103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-5937c00c-a0af-4098-9f16-1f6a242380dd admin admin] The token cache is being invalidate because project b21cb16c87864cd2bb49e616698e135c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:08.278444 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-5937c00c-a0af-4098-9f16-1f6a242380dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:08.279209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-5937c00c-a0af-4098-9f16-1f6a242380dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:08.280621 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d63879ac-48f3-47ce-be3d-01b1b2680eb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f021e1ce06a41a494fa261156b871e3', 'request_id': 'req-d63879ac-48f3-47ce-be3d-01b1b2680eb6'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:08.280816 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 984/1973] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:08 2022] DELETE /identity/v3/projects/b21cb16c87864cd2bb49e616698e135c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:08.285674 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714f3ef-bdd4-4584-b425-d01fccd3911d None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.285955 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714f3ef-bdd4-4584-b425-d01fccd3911d None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.286227 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714f3ef-bdd4-4584-b425-d01fccd3911d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.289191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d63879ac-48f3-47ce-be3d-01b1b2680eb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f021e1ce06a41a494fa261156b871e3', 'request_id': 'req-d63879ac-48f3-47ce-be3d-01b1b2680eb6'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:08.300742 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d63879ac-48f3-47ce-be3d-01b1b2680eb6 admin admin] The token cache is being invalidate because project 1f021e1ce06a41a494fa261156b871e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:08.301462 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d63879ac-48f3-47ce-be3d-01b1b2680eb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:08.302181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d63879ac-48f3-47ce-be3d-01b1b2680eb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:08.304071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 990/1974] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:08 2022] DELETE /identity/v3/projects/1f021e1ce06a41a494fa261156b871e3 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:08.306966 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-b714f3ef-bdd4-4584-b425-d01fccd3911d None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:08.308361 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b714f3ef-bdd4-4584-b425-d01fccd3911d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:08.308993 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ca16d-3489-4bb8-a0b2-521b33796d16 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.309283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ca16d-3489-4bb8-a0b2-521b33796d16 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.309684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ca16d-3489-4bb8-a0b2-521b33796d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.330670 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-ed4ca16d-3489-4bb8-a0b2-521b33796d16 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:08.331289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ed4ca16d-3489-4bb8-a0b2-521b33796d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:08.378389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 985/1975] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:08.387353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c5853-4794-4f56-8537-85a46c078d07 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.387840 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c5853-4794-4f56-8537-85a46c078d07 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.388259 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c5853-4794-4f56-8537-85a46c078d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.411849 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 991/1976] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:08.414178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796be7e-6e26-4906-bb13-3cb0ba994bbb None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.414610 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796be7e-6e26-4906-bb13-3cb0ba994bbb None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.415167 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796be7e-6e26-4906-bb13-3cb0ba994bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.417210 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-603c5853-4794-4f56-8537-85a46c078d07 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:08.419151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-603c5853-4794-4f56-8537-85a46c078d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:08.429808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 986/1977] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:08.434585 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c5853-4794-4f56-8537-85a46c078d07 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:08.440140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-4796be7e-6e26-4906-bb13-3cb0ba994bbb None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:08.441358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-4796be7e-6e26-4906-bb13-3cb0ba994bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:08.444236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c5853-4794-4f56-8537-85a46c078d07 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:08.448892 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3969c1-ca3e-4397-a6bd-2cbb762c1abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:08.449491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3969c1-ca3e-4397-a6bd-2cbb762c1abf admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.449659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 992/1978] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:08.449778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3969c1-ca3e-4397-a6bd-2cbb762c1abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.450026 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3969c1-ca3e-4397-a6bd-2cbb762c1abf admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.450665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3969c1-ca3e-4397-a6bd-2cbb762c1abf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:08.450992 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3969c1-ca3e-4397-a6bd-2cbb762c1abf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:08.452613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3969c1-ca3e-4397-a6bd-2cbb762c1abf admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:08.453821 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796be7e-6e26-4906-bb13-3cb0ba994bbb None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:08.459506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:08.469943 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 987/1979] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:59:08.470132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796be7e-6e26-4906-bb13-3cb0ba994bbb None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:08.471235 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3969c1-ca3e-4397-a6bd-2cbb762c1abf admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:08.473621 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1feea-9ce2-45f0-a905-82a2acfd4818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:08.474390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1feea-9ce2-45f0-a905-82a2acfd4818 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.474671 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1feea-9ce2-45f0-a905-82a2acfd4818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.474974 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1feea-9ce2-45f0-a905-82a2acfd4818 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.475620 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1feea-9ce2-45f0-a905-82a2acfd4818 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:08.475927 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1feea-9ce2-45f0-a905-82a2acfd4818 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:08.477558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1feea-9ce2-45f0-a905-82a2acfd4818 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:08.482287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3969c1-ca3e-4397-a6bd-2cbb762c1abf admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:08.486973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a38d6-950e-4706-9e7f-c1abe127226e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:08.487568 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a38d6-950e-4706-9e7f-c1abe127226e admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.487845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a38d6-950e-4706-9e7f-c1abe127226e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.488121 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a38d6-950e-4706-9e7f-c1abe127226e admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.488830 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a38d6-950e-4706-9e7f-c1abe127226e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:08.493250 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 993/1980] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:59:08.493388 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a38d6-950e-4706-9e7f-c1abe127226e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:08.494245 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1feea-9ce2-45f0-a905-82a2acfd4818 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:08.503891 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1feea-9ce2-45f0-a905-82a2acfd4818 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:08.509838 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2bf864-a527-4410-b595-a869ce64de4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:08.510452 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2bf864-a527-4410-b595-a869ce64de4f admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.510770 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2bf864-a527-4410-b595-a869ce64de4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.511138 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2bf864-a527-4410-b595-a869ce64de4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.511771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2bf864-a527-4410-b595-a869ce64de4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:08.513670 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2bf864-a527-4410-b595-a869ce64de4f admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:08.521652 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 988/1981] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:08 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:08.522964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a38d6-950e-4706-9e7f-c1abe127226e admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:08.549534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 994/1982] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:08 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:08.553458 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2bf864-a527-4410-b595-a869ce64de4f admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:08.606704 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a38d6-950e-4706-9e7f-c1abe127226e admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:08.610389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:08.625324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2bf864-a527-4410-b595-a869ce64de4f admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:08.630588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e643ed1-1b71-4c12-adce-c12f26222431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:08.631173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e643ed1-1b71-4c12-adce-c12f26222431 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.631453 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e643ed1-1b71-4c12-adce-c12f26222431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.631780 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e643ed1-1b71-4c12-adce-c12f26222431 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.632414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e643ed1-1b71-4c12-adce-c12f26222431 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:08.634422 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e643ed1-1b71-4c12-adce-c12f26222431 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:08.657909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 995/1983] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:08 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:08.664215 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e643ed1-1b71-4c12-adce-c12f26222431 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:08.674205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17606e-c541-41d3-a0d3-af9892887944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:08.675042 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17606e-c541-41d3-a0d3-af9892887944 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.675321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17606e-c541-41d3-a0d3-af9892887944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.675593 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17606e-c541-41d3-a0d3-af9892887944 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.676221 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17606e-c541-41d3-a0d3-af9892887944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:08.678440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e643ed1-1b71-4c12-adce-c12f26222431 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:08.678538 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17606e-c541-41d3-a0d3-af9892887944 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:08.683971 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17ad19-db8b-4072-99f5-457127fcf38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:08.684594 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17ad19-db8b-4072-99f5-457127fcf38f admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.684874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17ad19-db8b-4072-99f5-457127fcf38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.685148 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17ad19-db8b-4072-99f5-457127fcf38f admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.685801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17ad19-db8b-4072-99f5-457127fcf38f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:08.687634 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17ad19-db8b-4072-99f5-457127fcf38f admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:08.705586 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 996/1984] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:08.707318 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17ad19-db8b-4072-99f5-457127fcf38f admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:08.711104 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 989/1985] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:08 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:08.715169 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17606e-c541-41d3-a0d3-af9892887944 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:08.719141 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17ad19-db8b-4072-99f5-457127fcf38f admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:08.724605 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238e4ad-3616-40ab-842d-f922ea0d973b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:08.725491 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238e4ad-3616-40ab-842d-f922ea0d973b admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.725949 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238e4ad-3616-40ab-842d-f922ea0d973b admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.726545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238e4ad-3616-40ab-842d-f922ea0d973b admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.727461 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238e4ad-3616-40ab-842d-f922ea0d973b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:08.729490 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17606e-c541-41d3-a0d3-af9892887944 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:08.730230 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238e4ad-3616-40ab-842d-f922ea0d973b admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:08.735959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014cacd4-5457-49ea-a7be-379bd9e6e9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:08.736687 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cacd4-5457-49ea-a7be-379bd9e6e9ce admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.736967 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cacd4-5457-49ea-a7be-379bd9e6e9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.737304 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cacd4-5457-49ea-a7be-379bd9e6e9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.738096 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014cacd4-5457-49ea-a7be-379bd9e6e9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:08.739786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014cacd4-5457-49ea-a7be-379bd9e6e9ce admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:08.747857 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 997/1986] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:08.750417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238e4ad-3616-40ab-842d-f922ea0d973b admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:08.755346 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 990/1987] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:59:08.758907 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014cacd4-5457-49ea-a7be-379bd9e6e9ce admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:08.761454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238e4ad-3616-40ab-842d-f922ea0d973b admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:08.771091 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014cacd4-5457-49ea-a7be-379bd9e6e9ce admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:08.771386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d9b9a-4455-4769-9593-29d255cdf45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:08.773663 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d9b9a-4455-4769-9593-29d255cdf45e admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.773923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d9b9a-4455-4769-9593-29d255cdf45e admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.774164 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d9b9a-4455-4769-9593-29d255cdf45e admin admin] PATH_INFO: `/v3/projects/6529b71db4724a7ba43a06155030894f/users/d9e42e0dcb4e43fd92a3dc72e67ca04f/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.778567 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cdd3d8-9e33-4b42-ba54-ce9f0fd1bcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:08.779550 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdd3d8-9e33-4b42-ba54-ce9f0fd1bcd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.779917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdd3d8-9e33-4b42-ba54-ce9f0fd1bcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.780345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cdd3d8-9e33-4b42-ba54-ce9f0fd1bcd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.781154 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cdd3d8-9e33-4b42-ba54-ce9f0fd1bcd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:08.784370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cdd3d8-9e33-4b42-ba54-ce9f0fd1bcd0 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:08.799706 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 991/1988] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:59:08.803104 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d9b9a-4455-4769-9593-29d255cdf45e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6529b71db4724a7ba43a06155030894f, user_id=d9e42e0dcb4e43fd92a3dc72e67ca04f, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:08.803588 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cdd3d8-9e33-4b42-ba54-ce9f0fd1bcd0 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:08.806315 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d9b9a-4455-4769-9593-29d255cdf45e admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:08.823040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cdd3d8-9e33-4b42-ba54-ce9f0fd1bcd0 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:08.837996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bcaeb4-e515-436d-9213-b88d292ede0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:08.838806 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcaeb4-e515-436d-9213-b88d292ede0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.839102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcaeb4-e515-436d-9213-b88d292ede0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.839384 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcaeb4-e515-436d-9213-b88d292ede0f admin admin] PATH_INFO: `/v3/projects/718841495007434a8295beec5315fee6/users/cc621dbc2db943f08957c1903471fba7/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.846848 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 998/1989] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:08 2022] PUT /identity/v3/projects/6529b71db4724a7ba43a06155030894f/users/d9e42e0dcb4e43fd92a3dc72e67ca04f/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:08.850287 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66908db0-d386-4fa4-ac49-9d7cbfc64796 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.850680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66908db0-d386-4fa4-ac49-9d7cbfc64796 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.851252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66908db0-d386-4fa4-ac49-9d7cbfc64796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.872786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bcaeb4-e515-436d-9213-b88d292ede0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=718841495007434a8295beec5315fee6, user_id=cc621dbc2db943f08957c1903471fba7, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:08.876186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bcaeb4-e515-436d-9213-b88d292ede0f admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:08.890314 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-66908db0-d386-4fa4-ac49-9d7cbfc64796 None None] MFA Rules not processed for user `d9e42e0dcb4e43fd92a3dc72e67ca04f`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:08.891678 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-66908db0-d386-4fa4-ac49-9d7cbfc64796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:08.901903 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 992/1990] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:08 2022] PUT /identity/v3/projects/718841495007434a8295beec5315fee6/users/cc621dbc2db943f08957c1903471fba7/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:08.903994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd1b3a-2174-402b-a755-6fcce42ab091 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:08.904219 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd1b3a-2174-402b-a755-6fcce42ab091 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:08.904535 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd1b3a-2174-402b-a755-6fcce42ab091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:08.944382 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-73bd1b3a-2174-402b-a755-6fcce42ab091 None None] MFA Rules not processed for user `cc621dbc2db943f08957c1903471fba7`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:08.946071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-73bd1b3a-2174-402b-a755-6fcce42ab091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:09.003176 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:09.015237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 999/1991] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:09.037133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66908db0-d386-4fa4-ac49-9d7cbfc64796 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:09.042275 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-66908db0-d386-4fa4-ac49-9d7cbfc64796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:09.091804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 993/1992] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:09.096611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd1b3a-2174-402b-a755-6fcce42ab091 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:09.153293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66908db0-d386-4fa4-ac49-9d7cbfc64796 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:09.159317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4448f-e464-43ba-92f9-8a7f39a5aacd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:09.159700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4448f-e464-43ba-92f9-8a7f39a5aacd service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:09.160100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4448f-e464-43ba-92f9-8a7f39a5aacd service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:09.160462 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba4448f-e464-43ba-92f9-8a7f39a5aacd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:09.175546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4448f-e464-43ba-92f9-8a7f39a5aacd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:09.177165 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba4448f-e464-43ba-92f9-8a7f39a5aacd service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:09.182723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:09.190508 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1000/1993] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:09.212085 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4448f-e464-43ba-92f9-8a7f39a5aacd service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:09.255159 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd1b3a-2174-402b-a755-6fcce42ab091 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:09.260442 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605e98c-a521-4739-9230-05d86acd633d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:09.261564 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605e98c-a521-4739-9230-05d86acd633d admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:09.261896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605e98c-a521-4739-9230-05d86acd633d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:09.262402 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605e98c-a521-4739-9230-05d86acd633d admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:09.263580 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605e98c-a521-4739-9230-05d86acd633d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:09.265838 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605e98c-a521-4739-9230-05d86acd633d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:09.288097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba4448f-e464-43ba-92f9-8a7f39a5aacd service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:09.294672 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddd08c-8944-4ee1-bbaf-d96e0fdaf15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:09.295420 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddd08c-8944-4ee1-bbaf-d96e0fdaf15f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:09.295784 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddd08c-8944-4ee1-bbaf-d96e0fdaf15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:09.296063 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddd08c-8944-4ee1-bbaf-d96e0fdaf15f admin admin] PATH_INFO: `/v3/users/d9e42e0dcb4e43fd92a3dc72e67ca04f` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:09.297239 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ddd08c-8944-4ee1-bbaf-d96e0fdaf15f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9e42e0dcb4e43fd92a3dc72e67ca04f)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:09.299196 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ddd08c-8944-4ee1-bbaf-d96e0fdaf15f admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:09.313403 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 994/1994] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:09 2022] POST /identity/v3/projects => generated 381 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:09.314560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605e98c-a521-4739-9230-05d86acd633d admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:09.376323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-74ddd08c-8944-4ee1-bbaf-d96e0fdaf15f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9e42e0dcb4e43fd92a3dc72e67ca04f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:09.385005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-74ddd08c-8944-4ee1-bbaf-d96e0fdaf15f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9e42e0dcb4e43fd92a3dc72e67ca04f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:09.385998 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-74ddd08c-8944-4ee1-bbaf-d96e0fdaf15f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9e42e0dcb4e43fd92a3dc72e67ca04f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:09.391198 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605e98c-a521-4739-9230-05d86acd633d admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:09.396772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa349510-db3a-451a-b3b9-8b88485abf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:09.397493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa349510-db3a-451a-b3b9-8b88485abf63 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:09.397790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa349510-db3a-451a-b3b9-8b88485abf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:09.398047 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa349510-db3a-451a-b3b9-8b88485abf63 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:09.398675 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa349510-db3a-451a-b3b9-8b88485abf63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:09.400833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-74ddd08c-8944-4ee1-bbaf-d96e0fdaf15f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9e42e0dcb4e43fd92a3dc72e67ca04f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:09.401291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa349510-db3a-451a-b3b9-8b88485abf63 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:09.401525 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-74ddd08c-8944-4ee1-bbaf-d96e0fdaf15f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9e42e0dcb4e43fd92a3dc72e67ca04f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:09.415422 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1001/1995] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:09 2022] DELETE /identity/v3/users/d9e42e0dcb4e43fd92a3dc72e67ca04f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:09.419997 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04eadb-c7dc-4b23-b020-5203c537986a None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:09.420288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04eadb-c7dc-4b23-b020-5203c537986a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:09.420563 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04eadb-c7dc-4b23-b020-5203c537986a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:09.426808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 995/1996] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:09 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:09.428020 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa349510-db3a-451a-b3b9-8b88485abf63 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:09.429872 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-fa349510-db3a-451a-b3b9-8b88485abf63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:09.440892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-cc04eadb-c7dc-4b23-b020-5203c537986a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:09.442098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-cc04eadb-c7dc-4b23-b020-5203c537986a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:09.442511 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa349510-db3a-451a-b3b9-8b88485abf63 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:09.461271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:09.511481 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1002/1997] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:09.522009 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04eadb-c7dc-4b23-b020-5203c537986a None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:09.523413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-cc04eadb-c7dc-4b23-b020-5203c537986a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:09.526830 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f3ffa-d4c5-4ef7-9468-d6fef0a5b82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:09.527463 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f3ffa-d4c5-4ef7-9468-d6fef0a5b82b admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:09.527743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f3ffa-d4c5-4ef7-9468-d6fef0a5b82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:09.528089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f3ffa-d4c5-4ef7-9468-d6fef0a5b82b admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:09.528886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f3ffa-d4c5-4ef7-9468-d6fef0a5b82b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:09.531418 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5f3ffa-d4c5-4ef7-9468-d6fef0a5b82b admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:09.547809 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 996/1998] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:09.552053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f3ffa-d4c5-4ef7-9468-d6fef0a5b82b admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:09.564572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f3ffa-d4c5-4ef7-9468-d6fef0a5b82b admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:09.569468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dad95-e7e6-4d74-89a5-09a47f9c5134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:09.570161 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dad95-e7e6-4d74-89a5-09a47f9c5134 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:09.570443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dad95-e7e6-4d74-89a5-09a47f9c5134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:09.570720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7dad95-e7e6-4d74-89a5-09a47f9c5134 admin admin] PATH_INFO: `/v3/projects/e83d3e12fbb944ee867471e1e4ada2f3/users/cc009dada84c4a38b6216d691dfae470/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:09.588465 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7dad95-e7e6-4d74-89a5-09a47f9c5134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e83d3e12fbb944ee867471e1e4ada2f3, user_id=cc009dada84c4a38b6216d691dfae470, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:09.591774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7dad95-e7e6-4d74-89a5-09a47f9c5134 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:09.599311 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04eadb-c7dc-4b23-b020-5203c537986a None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:09.611458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 997/1999] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:09 2022] PUT /identity/v3/projects/e83d3e12fbb944ee867471e1e4ada2f3/users/cc009dada84c4a38b6216d691dfae470/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:09.615251 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6361e-77ba-4dac-a526-117030e6a0a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:09.615768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dad95-e7e6-4d74-89a5-09a47f9c5134 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:09.616018 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6361e-77ba-4dac-a526-117030e6a0a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:09.616356 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6361e-77ba-4dac-a526-117030e6a0a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:09.616933 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6361e-77ba-4dac-a526-117030e6a0a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:09.632798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6361e-77ba-4dac-a526-117030e6a0a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:09.634354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6361e-77ba-4dac-a526-117030e6a0a3 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:09.649104 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1003/2000] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:09.661125 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:09.712525 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7dad95-e7e6-4d74-89a5-09a47f9c5134 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:09.721310 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f7102-9927-401f-bc89-b9f43eaf2d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:09.722021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f7102-9927-401f-bc89-b9f43eaf2d2d admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:09.722372 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f7102-9927-401f-bc89-b9f43eaf2d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:09.722808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f7102-9927-401f-bc89-b9f43eaf2d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:09.724510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f7102-9927-401f-bc89-b9f43eaf2d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:09.726585 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f7102-9927-401f-bc89-b9f43eaf2d2d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:09.745390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 998/2001] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:59:09.746864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6361e-77ba-4dac-a526-117030e6a0a3 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:09.761015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6361e-77ba-4dac-a526-117030e6a0a3 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:09.768554 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca7506-0f50-47c9-8f3f-35387893f2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:09.769624 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca7506-0f50-47c9-8f3f-35387893f2bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:09.770014 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca7506-0f50-47c9-8f3f-35387893f2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:09.770447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca7506-0f50-47c9-8f3f-35387893f2bf admin admin] PATH_INFO: `/v3/projects/e83d3e12fbb944ee867471e1e4ada2f3/users/cc009dada84c4a38b6216d691dfae470/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:09.773540 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca7506-0f50-47c9-8f3f-35387893f2bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e83d3e12fbb944ee867471e1e4ada2f3, user_id=cc009dada84c4a38b6216d691dfae470, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:09.778530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca7506-0f50-47c9-8f3f-35387893f2bf admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:09.797048 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1004/2002] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:09 2022] PUT /identity/v3/projects/e83d3e12fbb944ee867471e1e4ada2f3/users/cc009dada84c4a38b6216d691dfae470/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:09.808806 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0810ba-d247-4d1d-95e4-b6c178be16b9 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:09.809103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0810ba-d247-4d1d-95e4-b6c178be16b9 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:09.809678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0810ba-d247-4d1d-95e4-b6c178be16b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:09.849875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-4d0810ba-d247-4d1d-95e4-b6c178be16b9 None None] MFA Rules not processed for user `cc009dada84c4a38b6216d691dfae470`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:09.852659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4d0810ba-d247-4d1d-95e4-b6c178be16b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:09.960996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 999/2003] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:09.966497 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca7506-0f50-47c9-8f3f-35387893f2bf admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:09.969202 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7eca7506-0f50-47c9-8f3f-35387893f2bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:09.973566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0810ba-d247-4d1d-95e4-b6c178be16b9 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:09.974820 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4d0810ba-d247-4d1d-95e4-b6c178be16b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:10.085523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca7506-0f50-47c9-8f3f-35387893f2bf admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:10.091872 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361635a-665d-444c-8aa1-6340d9a485d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:10.092529 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361635a-665d-444c-8aa1-6340d9a485d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:10.092882 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361635a-665d-444c-8aa1-6340d9a485d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:10.093155 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361635a-665d-444c-8aa1-6340d9a485d2 admin admin] PATH_INFO: `/v3/users/94a910c81fa24e5e88ffda6cc9b86100` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:10.094353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361635a-665d-444c-8aa1-6340d9a485d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94a910c81fa24e5e88ffda6cc9b86100)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:10.096359 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361635a-665d-444c-8aa1-6340d9a485d2 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:10.109690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0810ba-d247-4d1d-95e4-b6c178be16b9 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:10.114920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa1f70-7135-4ca2-a534-cf896610dedd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:10.115796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa1f70-7135-4ca2-a534-cf896610dedd service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:10.116390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa1f70-7135-4ca2-a534-cf896610dedd service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:10.116939 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa1f70-7135-4ca2-a534-cf896610dedd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:10.130442 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa1f70-7135-4ca2-a534-cf896610dedd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:10.132112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa1f70-7135-4ca2-a534-cf896610dedd service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:10.162923 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1000/2004] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:59:10.163939 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa1f70-7135-4ca2-a534-cf896610dedd service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:10.165343 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-35fa1f70-7135-4ca2-a534-cf896610dedd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:10.179227 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b361635a-665d-444c-8aa1-6340d9a485d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94a910c81fa24e5e88ffda6cc9b86100'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:10.184846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa1f70-7135-4ca2-a534-cf896610dedd service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:10.191611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b361635a-665d-444c-8aa1-6340d9a485d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94a910c81fa24e5e88ffda6cc9b86100'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:10.192286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b361635a-665d-444c-8aa1-6340d9a485d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94a910c81fa24e5e88ffda6cc9b86100'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:10.196737 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:10.200549 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373bc511-2bf2-42a0-8e27-756947817439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:10.201413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bc511-2bf2-42a0-8e27-756947817439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:10.201699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bc511-2bf2-42a0-8e27-756947817439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:10.202014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373bc511-2bf2-42a0-8e27-756947817439 admin admin] PATH_INFO: `/v3/projects/6529b71db4724a7ba43a06155030894f` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:10.208312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bc511-2bf2-42a0-8e27-756947817439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6529b71db4724a7ba43a06155030894f)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:10.211282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373bc511-2bf2-42a0-8e27-756947817439 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:10.217807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b361635a-665d-444c-8aa1-6340d9a485d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94a910c81fa24e5e88ffda6cc9b86100'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:10.218887 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b361635a-665d-444c-8aa1-6340d9a485d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94a910c81fa24e5e88ffda6cc9b86100'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:10.246048 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1005/2005] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:09 2022] DELETE /identity/v3/users/94a910c81fa24e5e88ffda6cc9b86100 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:10.247513 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361635a-665d-444c-8aa1-6340d9a485d2 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:10.251995 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b361635a-665d-444c-8aa1-6340d9a485d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:10.277075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:10.283827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-373bc511-2bf2-42a0-8e27-756947817439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6529b71db4724a7ba43a06155030894f', 'request_id': 'req-373bc511-2bf2-42a0-8e27-756947817439'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:10.291980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-373bc511-2bf2-42a0-8e27-756947817439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6529b71db4724a7ba43a06155030894f', 'request_id': 'req-373bc511-2bf2-42a0-8e27-756947817439'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:10.298503 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-373bc511-2bf2-42a0-8e27-756947817439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6529b71db4724a7ba43a06155030894f', 'request_id': 'req-373bc511-2bf2-42a0-8e27-756947817439'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:10.317151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-373bc511-2bf2-42a0-8e27-756947817439 admin admin] The token cache is being invalidate because project 6529b71db4724a7ba43a06155030894f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:10.317447 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-373bc511-2bf2-42a0-8e27-756947817439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:10.318186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-373bc511-2bf2-42a0-8e27-756947817439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:10.323827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1001/2006] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:10 2022] DELETE /identity/v3/projects/6529b71db4724a7ba43a06155030894f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:10.331390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956ad6d-7285-40ba-b89e-73caa29a74d7 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:10.331544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956ad6d-7285-40ba-b89e-73caa29a74d7 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:10.334044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956ad6d-7285-40ba-b89e-73caa29a74d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:10.354959 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361635a-665d-444c-8aa1-6340d9a485d2 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:10.359717 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-1956ad6d-7285-40ba-b89e-73caa29a74d7 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:10.361025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1956ad6d-7285-40ba-b89e-73caa29a74d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:10.438511 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423db980-65b4-4d23-9e08-c39b7c17ea5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:10.439445 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423db980-65b4-4d23-9e08-c39b7c17ea5c service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:10.440001 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423db980-65b4-4d23-9e08-c39b7c17ea5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:10.440292 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423db980-65b4-4d23-9e08-c39b7c17ea5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:10.442900 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-423db980-65b4-4d23-9e08-c39b7c17ea5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:10.454190 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1002/2007] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:10.455484 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ae761-9ab0-42e4-ba8f-818bd3f5f580 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:10.455767 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ae761-9ab0-42e4-ba8f-818bd3f5f580 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:10.456036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ae761-9ab0-42e4-ba8f-818bd3f5f580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:10.485237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-b30ae761-9ab0-42e4-ba8f-818bd3f5f580 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:10.488351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b30ae761-9ab0-42e4-ba8f-818bd3f5f580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:10.524083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1003/2008] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:10.527455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ae761-9ab0-42e4-ba8f-818bd3f5f580 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:10.531526 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b30ae761-9ab0-42e4-ba8f-818bd3f5f580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:10.571865 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423db980-65b4-4d23-9e08-c39b7c17ea5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:10.575515 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423db980-65b4-4d23-9e08-c39b7c17ea5c service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:10.599060 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1006/2009] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:10.601213 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f6b6a-579b-4c63-98e4-9507bd1fb694 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:10.601567 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f6b6a-579b-4c63-98e4-9507bd1fb694 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:10.601849 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f6b6a-579b-4c63-98e4-9507bd1fb694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:10.633657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-d02f6b6a-579b-4c63-98e4-9507bd1fb694 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:10.635470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d02f6b6a-579b-4c63-98e4-9507bd1fb694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:10.661710 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ae761-9ab0-42e4-ba8f-818bd3f5f580 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:10.667269 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc080e-9339-4f3d-8dc7-db75b871e038 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:10.667369 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1007/2010] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:10.667460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc080e-9339-4f3d-8dc7-db75b871e038 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:10.667821 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc080e-9339-4f3d-8dc7-db75b871e038 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:10.668160 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc080e-9339-4f3d-8dc7-db75b871e038 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:10.669484 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f6b6a-579b-4c63-98e4-9507bd1fb694 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:10.680657 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc080e-9339-4f3d-8dc7-db75b871e038 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:10.680657 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc080e-9339-4f3d-8dc7-db75b871e038 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:10.692658 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1004/2011] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:10.694331 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f6b6a-579b-4c63-98e4-9507bd1fb694 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:10.699587 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a1ae5-dce3-4401-b374-7b7e0e2050e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:10.700448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a1ae5-dce3-4401-b374-7b7e0e2050e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:10.701246 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a1ae5-dce3-4401-b374-7b7e0e2050e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:10.701693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a1ae5-dce3-4401-b374-7b7e0e2050e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:10.702934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a1ae5-dce3-4401-b374-7b7e0e2050e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:10.703400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a1ae5-dce3-4401-b374-7b7e0e2050e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:10.705260 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a1ae5-dce3-4401-b374-7b7e0e2050e4 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:10.725874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1008/2012] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:10.733505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc080e-9339-4f3d-8dc7-db75b871e038 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:10.745824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc080e-9339-4f3d-8dc7-db75b871e038 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:10.756698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b41c01-ca45-4aaf-8f02-859256d02074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:10.757581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b41c01-ca45-4aaf-8f02-859256d02074 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:10.758079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b41c01-ca45-4aaf-8f02-859256d02074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:10.758558 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b41c01-ca45-4aaf-8f02-859256d02074 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:10.759545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b41c01-ca45-4aaf-8f02-859256d02074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:10.762382 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b41c01-ca45-4aaf-8f02-859256d02074 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:10.819258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1005/2013] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:10 2022] POST /identity/v3/projects => generated 383 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:10.825761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a1ae5-dce3-4401-b374-7b7e0e2050e4 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:10.838458 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a1ae5-dce3-4401-b374-7b7e0e2050e4 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:10.872534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:10.936773 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66154c-4759-4cf7-bf60-2840084503fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:10.937811 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66154c-4759-4cf7-bf60-2840084503fa admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:10.938252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66154c-4759-4cf7-bf60-2840084503fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:10.939072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66154c-4759-4cf7-bf60-2840084503fa admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:10.942280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66154c-4759-4cf7-bf60-2840084503fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:10.950357 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66154c-4759-4cf7-bf60-2840084503fa admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:11.007107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b41c01-ca45-4aaf-8f02-859256d02074 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:11.015466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1009/2014] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:10 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:11.023493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b41c01-ca45-4aaf-8f02-859256d02074 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:11.061197 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734ffd8-8e0a-4349-a3b3-cdb652402d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:11.065049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734ffd8-8e0a-4349-a3b3-cdb652402d88 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:11.065156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734ffd8-8e0a-4349-a3b3-cdb652402d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:11.065433 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734ffd8-8e0a-4349-a3b3-cdb652402d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:11.066103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734ffd8-8e0a-4349-a3b3-cdb652402d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:11.067761 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734ffd8-8e0a-4349-a3b3-cdb652402d88 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:11.104555 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1006/2015] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:11.110875 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66154c-4759-4cf7-bf60-2840084503fa admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:11.131166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66154c-4759-4cf7-bf60-2840084503fa admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:11.145534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd41cbd-5dd8-428f-bce6-c0dd706a9ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:11.146215 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd41cbd-5dd8-428f-bce6-c0dd706a9ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:11.146494 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd41cbd-5dd8-428f-bce6-c0dd706a9ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:11.146793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd41cbd-5dd8-428f-bce6-c0dd706a9ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:11.147512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd41cbd-5dd8-428f-bce6-c0dd706a9ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:11.151463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd41cbd-5dd8-428f-bce6-c0dd706a9ce8 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:11.176723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1010/2016] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:11.181118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734ffd8-8e0a-4349-a3b3-cdb652402d88 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:11.193237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734ffd8-8e0a-4349-a3b3-cdb652402d88 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:11.205684 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82185e28-55fc-4c71-ad40-b0537d218697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:11.206370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82185e28-55fc-4c71-ad40-b0537d218697 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:11.207809 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82185e28-55fc-4c71-ad40-b0537d218697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:11.207809 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82185e28-55fc-4c71-ad40-b0537d218697 admin admin] PATH_INFO: `/v3/projects/839309f2f3404355a43f2c0664445e44/users/fa9b638bc1414e85ae4c4c30c95effc1/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:11.217163 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd41cbd-5dd8-428f-bce6-c0dd706a9ce8 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:11.219070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-afd41cbd-5dd8-428f-bce6-c0dd706a9ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:11.236646 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd41cbd-5dd8-428f-bce6-c0dd706a9ce8 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:11.242854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b4352-7fbf-46d0-a171-04cc1559409f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:11.244426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b4352-7fbf-46d0-a171-04cc1559409f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:11.244902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b4352-7fbf-46d0-a171-04cc1559409f admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:11.245325 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b4352-7fbf-46d0-a171-04cc1559409f admin admin] PATH_INFO: `/v3/users/cc621dbc2db943f08957c1903471fba7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:11.247039 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b4352-7fbf-46d0-a171-04cc1559409f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc621dbc2db943f08957c1903471fba7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:11.249105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b4352-7fbf-46d0-a171-04cc1559409f admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:11.254667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82185e28-55fc-4c71-ad40-b0537d218697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=839309f2f3404355a43f2c0664445e44, user_id=fa9b638bc1414e85ae4c4c30c95effc1, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:11.257706 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82185e28-55fc-4c71-ad40-b0537d218697 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:11.285939 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1007/2017] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:11 2022] PUT /identity/v3/projects/839309f2f3404355a43f2c0664445e44/users/fa9b638bc1414e85ae4c4c30c95effc1/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:11.298317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e3bfc-ff0f-4626-88e2-a152d2a25532 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:11.298681 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e3bfc-ff0f-4626-88e2-a152d2a25532 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:11.299653 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e3bfc-ff0f-4626-88e2-a152d2a25532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:11.369276 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3e7b4352-7fbf-46d0-a171-04cc1559409f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc621dbc2db943f08957c1903471fba7'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:11.381410 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3e7b4352-7fbf-46d0-a171-04cc1559409f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc621dbc2db943f08957c1903471fba7'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:11.382604 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3e7b4352-7fbf-46d0-a171-04cc1559409f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc621dbc2db943f08957c1903471fba7'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:11.388321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:11.392641 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-322e3bfc-ff0f-4626-88e2-a152d2a25532 None None] MFA Rules not processed for user `fa9b638bc1414e85ae4c4c30c95effc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:11.394840 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-322e3bfc-ff0f-4626-88e2-a152d2a25532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:11.398506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3e7b4352-7fbf-46d0-a171-04cc1559409f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc621dbc2db943f08957c1903471fba7'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:11.399383 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3e7b4352-7fbf-46d0-a171-04cc1559409f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc621dbc2db943f08957c1903471fba7'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:11.414780 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1011/2018] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:11 2022] DELETE /identity/v3/users/cc621dbc2db943f08957c1903471fba7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:11.415935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b4352-7fbf-46d0-a171-04cc1559409f admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:11.417494 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3e7b4352-7fbf-46d0-a171-04cc1559409f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:11.479138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1008/2019] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:11.479138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390d0ec-4f31-4252-b007-85c8329436c7 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:11.479138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390d0ec-4f31-4252-b007-85c8329436c7 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:11.479138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390d0ec-4f31-4252-b007-85c8329436c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:11.499752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-7390d0ec-4f31-4252-b007-85c8329436c7 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:11.501386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7390d0ec-4f31-4252-b007-85c8329436c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:11.502048 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b4352-7fbf-46d0-a171-04cc1559409f admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:11.506937 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570a6c9-bfb8-4531-abd2-96a88773c5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:11.507688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570a6c9-bfb8-4531-abd2-96a88773c5ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:11.508067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570a6c9-bfb8-4531-abd2-96a88773c5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:11.508462 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570a6c9-bfb8-4531-abd2-96a88773c5ca admin admin] PATH_INFO: `/v3/projects/23f8e1537a634f2f9cda7b9b4df0c135` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:11.510362 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570a6c9-bfb8-4531-abd2-96a88773c5ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=23f8e1537a634f2f9cda7b9b4df0c135)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:11.512437 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570a6c9-bfb8-4531-abd2-96a88773c5ca admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:11.516212 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1009/2020] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:11.516977 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390d0ec-4f31-4252-b007-85c8329436c7 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:11.518346 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7390d0ec-4f31-4252-b007-85c8329436c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:11.531846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390d0ec-4f31-4252-b007-85c8329436c7 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:11.532475 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:11.537416 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811a742-f6e9-45e2-bbab-26b0a1e34b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:11.538038 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811a742-f6e9-45e2-bbab-26b0a1e34b58 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:11.538319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811a742-f6e9-45e2-bbab-26b0a1e34b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:11.538593 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811a742-f6e9-45e2-bbab-26b0a1e34b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:11.539281 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811a742-f6e9-45e2-bbab-26b0a1e34b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:11.541246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811a742-f6e9-45e2-bbab-26b0a1e34b58 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:11.582506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1010/2021] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:11 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:11.582506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811a742-f6e9-45e2-bbab-26b0a1e34b58 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:11.583197 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-6811a742-f6e9-45e2-bbab-26b0a1e34b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:11.583669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-4570a6c9-bfb8-4531-abd2-96a88773c5ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23f8e1537a634f2f9cda7b9b4df0c135', 'request_id': 'req-4570a6c9-bfb8-4531-abd2-96a88773c5ca'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:11.602344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-4570a6c9-bfb8-4531-abd2-96a88773c5ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23f8e1537a634f2f9cda7b9b4df0c135', 'request_id': 'req-4570a6c9-bfb8-4531-abd2-96a88773c5ca'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:11.614636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-4570a6c9-bfb8-4531-abd2-96a88773c5ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23f8e1537a634f2f9cda7b9b4df0c135', 'request_id': 'req-4570a6c9-bfb8-4531-abd2-96a88773c5ca'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:11.654464 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-4570a6c9-bfb8-4531-abd2-96a88773c5ca admin admin] The token cache is being invalidate because project 23f8e1537a634f2f9cda7b9b4df0c135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:11.655170 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-4570a6c9-bfb8-4531-abd2-96a88773c5ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:11.656085 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-4570a6c9-bfb8-4531-abd2-96a88773c5ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:11.658493 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1012/2022] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:11 2022] DELETE /identity/v3/projects/23f8e1537a634f2f9cda7b9b4df0c135 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:11.663573 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570a6c9-bfb8-4531-abd2-96a88773c5ca admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:11.665225 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-4570a6c9-bfb8-4531-abd2-96a88773c5ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:11.730205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811a742-f6e9-45e2-bbab-26b0a1e34b58 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:11.735451 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1ffaf-4657-4861-a647-8ba2bfd165ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:11.736210 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1ffaf-4657-4861-a647-8ba2bfd165ab service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:11.736591 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1ffaf-4657-4861-a647-8ba2bfd165ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:11.736964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1ffaf-4657-4861-a647-8ba2bfd165ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:11.741891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-eeb1ffaf-4657-4861-a647-8ba2bfd165ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:11.814348 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570a6c9-bfb8-4531-abd2-96a88773c5ca admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:11.839036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1ffaf-4657-4861-a647-8ba2bfd165ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:11.841549 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1ffaf-4657-4861-a647-8ba2bfd165ab service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:11.854348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1011/2023] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:11.855350 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1ffaf-4657-4861-a647-8ba2bfd165ab service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:11.856959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-eeb1ffaf-4657-4861-a647-8ba2bfd165ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:11.872162 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1ffaf-4657-4861-a647-8ba2bfd165ab service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:11.878701 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67084bee-12c8-4b2e-b448-5ba931bace19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:11.879394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67084bee-12c8-4b2e-b448-5ba931bace19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:11.879670 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67084bee-12c8-4b2e-b448-5ba931bace19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:11.879941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67084bee-12c8-4b2e-b448-5ba931bace19 admin admin] PATH_INFO: `/v3/users/6d895fbe4ee146f9afbfcdc1af27af9a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:11.881092 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67084bee-12c8-4b2e-b448-5ba931bace19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d895fbe4ee146f9afbfcdc1af27af9a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:11.882142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04d01e-4be6-46c0-9e8d-1bfb10ea8410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:11.882833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04d01e-4be6-46c0-9e8d-1bfb10ea8410 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:11.883121 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67084bee-12c8-4b2e-b448-5ba931bace19 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:11.883235 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04d01e-4be6-46c0-9e8d-1bfb10ea8410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:11.883400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04d01e-4be6-46c0-9e8d-1bfb10ea8410 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:11.884046 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04d01e-4be6-46c0-9e8d-1bfb10ea8410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:11.886351 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04d01e-4be6-46c0-9e8d-1bfb10ea8410 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:11.916702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:11.923947 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1013/2024] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:11 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:11.929004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04d01e-4be6-46c0-9e8d-1bfb10ea8410 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:11.945611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04d01e-4be6-46c0-9e8d-1bfb10ea8410 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:11.951032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db08c8-6f80-4786-b23f-30d7c53854c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:11.951935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db08c8-6f80-4786-b23f-30d7c53854c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:11.952214 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db08c8-6f80-4786-b23f-30d7c53854c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:11.952535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db08c8-6f80-4786-b23f-30d7c53854c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:11.953700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db08c8-6f80-4786-b23f-30d7c53854c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:11.955598 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db08c8-6f80-4786-b23f-30d7c53854c9 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:11.960004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-67084bee-12c8-4b2e-b448-5ba931bace19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d895fbe4ee146f9afbfcdc1af27af9a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:11.972233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-67084bee-12c8-4b2e-b448-5ba931bace19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d895fbe4ee146f9afbfcdc1af27af9a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:11.973400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1014/2025] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:11.974503 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db08c8-6f80-4786-b23f-30d7c53854c9 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:11.984537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-67084bee-12c8-4b2e-b448-5ba931bace19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d895fbe4ee146f9afbfcdc1af27af9a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:11.985486 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db08c8-6f80-4786-b23f-30d7c53854c9 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:11.991953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4bc7b-5431-4718-adb4-3a6ca3314c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:11.992701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4bc7b-5431-4718-adb4-3a6ca3314c10 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:11.993010 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4bc7b-5431-4718-adb4-3a6ca3314c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:11.993333 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4bc7b-5431-4718-adb4-3a6ca3314c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:11.994079 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4bc7b-5431-4718-adb4-3a6ca3314c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:11.995879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4bc7b-5431-4718-adb4-3a6ca3314c10 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:12.006328 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-67084bee-12c8-4b2e-b448-5ba931bace19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d895fbe4ee146f9afbfcdc1af27af9a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:12.007133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-67084bee-12c8-4b2e-b448-5ba931bace19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d895fbe4ee146f9afbfcdc1af27af9a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:12.011656 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1015/2026] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:12.012604 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4bc7b-5431-4718-adb4-3a6ca3314c10 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:12.014500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a8b4bc7b-5431-4718-adb4-3a6ca3314c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:12.015423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1012/2027] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:11 2022] DELETE /identity/v3/users/6d895fbe4ee146f9afbfcdc1af27af9a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:12.028553 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4bc7b-5431-4718-adb4-3a6ca3314c10 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:12.047952 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:12.118023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fc87e-00cd-40e1-b541-1207b0970df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:12.119384 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fc87e-00cd-40e1-b541-1207b0970df0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:12.120086 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fc87e-00cd-40e1-b541-1207b0970df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:12.123703 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fc87e-00cd-40e1-b541-1207b0970df0 admin admin] PATH_INFO: `/v3/projects/32acdf41c87240b998366ec8d6ca513c/users/5eff3d4eadeb4a7f97805ce162aec3c2/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:12.159817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fc87e-00cd-40e1-b541-1207b0970df0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32acdf41c87240b998366ec8d6ca513c, user_id=5eff3d4eadeb4a7f97805ce162aec3c2, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:12.164063 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fc87e-00cd-40e1-b541-1207b0970df0 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:12.192316 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1016/2028] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:12 2022] PUT /identity/v3/projects/32acdf41c87240b998366ec8d6ca513c/users/5eff3d4eadeb4a7f97805ce162aec3c2/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:12.203918 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e0c59-132e-43b5-99b7-a4bd70141a55 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:12.204258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e0c59-132e-43b5-99b7-a4bd70141a55 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:12.205328 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e0c59-132e-43b5-99b7-a4bd70141a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:12.250049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-860e0c59-132e-43b5-99b7-a4bd70141a55 None None] MFA Rules not processed for user `5eff3d4eadeb4a7f97805ce162aec3c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:12.250448 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-860e0c59-132e-43b5-99b7-a4bd70141a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:12.378764 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1013/2029] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:12.382917 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fc87e-00cd-40e1-b541-1207b0970df0 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:12.395214 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e0c59-132e-43b5-99b7-a4bd70141a55 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:12.397834 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-860e0c59-132e-43b5-99b7-a4bd70141a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:12.473111 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fc87e-00cd-40e1-b541-1207b0970df0 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:12.480998 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37349059-21e6-44fa-baea-1ff845ed9be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:12.482594 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37349059-21e6-44fa-baea-1ff845ed9be1 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:12.483127 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37349059-21e6-44fa-baea-1ff845ed9be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:12.483407 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37349059-21e6-44fa-baea-1ff845ed9be1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:12.484308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37349059-21e6-44fa-baea-1ff845ed9be1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:12.486222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37349059-21e6-44fa-baea-1ff845ed9be1 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:12.527120 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e0c59-132e-43b5-99b7-a4bd70141a55 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:12.533474 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73161f53-373c-469e-a5cc-932f6c987be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:12.534156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73161f53-373c-469e-a5cc-932f6c987be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:12.534635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73161f53-373c-469e-a5cc-932f6c987be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:12.537307 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73161f53-373c-469e-a5cc-932f6c987be9 admin admin] PATH_INFO: `/v3/projects/718841495007434a8295beec5315fee6` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:12.538776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73161f53-373c-469e-a5cc-932f6c987be9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=718841495007434a8295beec5315fee6)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:12.542019 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73161f53-373c-469e-a5cc-932f6c987be9 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:12.557254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1017/2030] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:12 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:12.560004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37349059-21e6-44fa-baea-1ff845ed9be1 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:12.608262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:12.648220 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37349059-21e6-44fa-baea-1ff845ed9be1 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:12.650274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-73161f53-373c-469e-a5cc-932f6c987be9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '718841495007434a8295beec5315fee6', 'request_id': 'req-73161f53-373c-469e-a5cc-932f6c987be9'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:12.672356 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-73161f53-373c-469e-a5cc-932f6c987be9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '718841495007434a8295beec5315fee6', 'request_id': 'req-73161f53-373c-469e-a5cc-932f6c987be9'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:12.688635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-73161f53-373c-469e-a5cc-932f6c987be9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '718841495007434a8295beec5315fee6', 'request_id': 'req-73161f53-373c-469e-a5cc-932f6c987be9'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:12.711247 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:12.722794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-73161f53-373c-469e-a5cc-932f6c987be9 admin admin] The token cache is being invalidate because project 718841495007434a8295beec5315fee6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:12.723237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-73161f53-373c-469e-a5cc-932f6c987be9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:12.724210 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-73161f53-373c-469e-a5cc-932f6c987be9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:12.727529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1014/2031] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:12 2022] DELETE /identity/v3/projects/718841495007434a8295beec5315fee6 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:12.729278 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73161f53-373c-469e-a5cc-932f6c987be9 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:12.733416 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-73161f53-373c-469e-a5cc-932f6c987be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:12.775729 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d27514-6ba5-4491-9043-3b281a656620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:12.776610 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d27514-6ba5-4491-9043-3b281a656620 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:12.776959 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d27514-6ba5-4491-9043-3b281a656620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:12.777293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d27514-6ba5-4491-9043-3b281a656620 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:12.778002 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d27514-6ba5-4491-9043-3b281a656620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:12.780704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d27514-6ba5-4491-9043-3b281a656620 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:12.811399 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1018/2032] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:12 2022] POST /identity/v3/users => generated 299 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:12.816353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d27514-6ba5-4491-9043-3b281a656620 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:12.818792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b8d27514-6ba5-4491-9043-3b281a656620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:12.838082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d27514-6ba5-4491-9043-3b281a656620 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:12.845686 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddead0ea-f4bb-4ba4-be60-ed5eb80a82e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:12.846568 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddead0ea-f4bb-4ba4-be60-ed5eb80a82e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:12.846965 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddead0ea-f4bb-4ba4-be60-ed5eb80a82e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:12.847308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddead0ea-f4bb-4ba4-be60-ed5eb80a82e9 admin admin] PATH_INFO: `/v3/users/cc009dada84c4a38b6216d691dfae470` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:12.848651 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddead0ea-f4bb-4ba4-be60-ed5eb80a82e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc009dada84c4a38b6216d691dfae470)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:12.851213 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddead0ea-f4bb-4ba4-be60-ed5eb80a82e9 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:12.860770 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73161f53-373c-469e-a5cc-932f6c987be9 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:12.867729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2fdf7-1523-4512-b77a-a3fdd1571729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:12.868455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2fdf7-1523-4512-b77a-a3fdd1571729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:12.868780 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2fdf7-1523-4512-b77a-a3fdd1571729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:12.869219 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2fdf7-1523-4512-b77a-a3fdd1571729 admin admin] PATH_INFO: `/v3/projects/141b2d7b54714739b04d34677fc19494` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:12.870625 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2fdf7-1523-4512-b77a-a3fdd1571729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=141b2d7b54714739b04d34677fc19494)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:12.872675 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2fdf7-1523-4512-b77a-a3fdd1571729 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:12.913702 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ddead0ea-f4bb-4ba4-be60-ed5eb80a82e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc009dada84c4a38b6216d691dfae470'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:12.925765 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ddead0ea-f4bb-4ba4-be60-ed5eb80a82e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc009dada84c4a38b6216d691dfae470'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:12.926526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ddead0ea-f4bb-4ba4-be60-ed5eb80a82e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc009dada84c4a38b6216d691dfae470'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:12.943773 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ddead0ea-f4bb-4ba4-be60-ed5eb80a82e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc009dada84c4a38b6216d691dfae470'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:12.944548 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ddead0ea-f4bb-4ba4-be60-ed5eb80a82e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc009dada84c4a38b6216d691dfae470'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:12.956835 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-59e2fdf7-1523-4512-b77a-a3fdd1571729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '141b2d7b54714739b04d34677fc19494', 'request_id': 'req-59e2fdf7-1523-4512-b77a-a3fdd1571729'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:12.961150 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1019/2033] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:12 2022] DELETE /identity/v3/users/cc009dada84c4a38b6216d691dfae470 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:12.965614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddead0ea-f4bb-4ba4-be60-ed5eb80a82e9 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:12.965614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ddead0ea-f4bb-4ba4-be60-ed5eb80a82e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:12.967772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-59e2fdf7-1523-4512-b77a-a3fdd1571729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '141b2d7b54714739b04d34677fc19494', 'request_id': 'req-59e2fdf7-1523-4512-b77a-a3fdd1571729'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:12.974485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-59e2fdf7-1523-4512-b77a-a3fdd1571729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '141b2d7b54714739b04d34677fc19494', 'request_id': 'req-59e2fdf7-1523-4512-b77a-a3fdd1571729'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:12.992151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-59e2fdf7-1523-4512-b77a-a3fdd1571729 admin admin] The token cache is being invalidate because project 141b2d7b54714739b04d34677fc19494 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:12.992666 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-59e2fdf7-1523-4512-b77a-a3fdd1571729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:12.993463 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-59e2fdf7-1523-4512-b77a-a3fdd1571729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:12.998142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1015/2034] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:12 2022] DELETE /identity/v3/projects/141b2d7b54714739b04d34677fc19494 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:13.000044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2fdf7-1523-4512-b77a-a3fdd1571729 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:13.003979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-59e2fdf7-1523-4512-b77a-a3fdd1571729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:13.073951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:13.085079 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddead0ea-f4bb-4ba4-be60-ed5eb80a82e9 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:13.097152 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a1b52-6f23-469a-bf28-3df62feb8854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:13.098974 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a1b52-6f23-469a-bf28-3df62feb8854 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:13.099511 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a1b52-6f23-469a-bf28-3df62feb8854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:13.099945 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a1b52-6f23-469a-bf28-3df62feb8854 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:13.103378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a1b52-6f23-469a-bf28-3df62feb8854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:13.106854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a1b52-6f23-469a-bf28-3df62feb8854 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:13.112633 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2fdf7-1523-4512-b77a-a3fdd1571729 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:13.120581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f2a97-b33a-4051-b735-edc13bbef483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:13.122101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f2a97-b33a-4051-b735-edc13bbef483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:13.122568 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f2a97-b33a-4051-b735-edc13bbef483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:13.123046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f2a97-b33a-4051-b735-edc13bbef483 admin admin] PATH_INFO: `/v3/users/93e17e326ecd44539577014dc89e2083` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:13.123718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1020/2035] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:12 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:13.125151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f2a97-b33a-4051-b735-edc13bbef483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93e17e326ecd44539577014dc89e2083)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:13.127352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33f2a97-b33a-4051-b735-edc13bbef483 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:13.128698 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a1b52-6f23-469a-bf28-3df62feb8854 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:13.131701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3e8a1b52-6f23-469a-bf28-3df62feb8854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:13.153397 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a1b52-6f23-469a-bf28-3df62feb8854 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:13.160495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d8d96-e873-4b06-a2af-f89855f81eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:13.161185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d8d96-e873-4b06-a2af-f89855f81eb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:13.161773 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d8d96-e873-4b06-a2af-f89855f81eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:13.161773 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d8d96-e873-4b06-a2af-f89855f81eb1 admin admin] PATH_INFO: `/v3/projects/be988d2808974611b7afdf7b79eb31da/users/8db7a3227d3141229a1132bb2262334a/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:13.208405 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d8d96-e873-4b06-a2af-f89855f81eb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be988d2808974611b7afdf7b79eb31da, user_id=8db7a3227d3141229a1132bb2262334a, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:13.211839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d8d96-e873-4b06-a2af-f89855f81eb1 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:13.216671 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-e33f2a97-b33a-4051-b735-edc13bbef483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93e17e326ecd44539577014dc89e2083'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:13.236325 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1021/2036] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:13 2022] PUT /identity/v3/projects/be988d2808974611b7afdf7b79eb31da/users/8db7a3227d3141229a1132bb2262334a/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:13.242163 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d8d96-e873-4b06-a2af-f89855f81eb1 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:13.244262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-e33f2a97-b33a-4051-b735-edc13bbef483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93e17e326ecd44539577014dc89e2083'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:13.252755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-e33f2a97-b33a-4051-b735-edc13bbef483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93e17e326ecd44539577014dc89e2083'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:13.254370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:13.270690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-e33f2a97-b33a-4051-b735-edc13bbef483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93e17e326ecd44539577014dc89e2083'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:13.271398 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-e33f2a97-b33a-4051-b735-edc13bbef483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93e17e326ecd44539577014dc89e2083'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:13.284343 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1016/2037] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:12 2022] DELETE /identity/v3/users/93e17e326ecd44539577014dc89e2083 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:13.336062 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d8d96-e873-4b06-a2af-f89855f81eb1 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:13.380753 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f2a97-b33a-4051-b735-edc13bbef483 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:13.382274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e33f2a97-b33a-4051-b735-edc13bbef483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:13.436225 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ca59b-ae82-4beb-be37-6c2444b5bdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:13.436919 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ca59b-ae82-4beb-be37-6c2444b5bdbc admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:13.437516 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ca59b-ae82-4beb-be37-6c2444b5bdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:13.437516 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ca59b-ae82-4beb-be37-6c2444b5bdbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:13.438339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503ca59b-ae82-4beb-be37-6c2444b5bdbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:13.440337 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503ca59b-ae82-4beb-be37-6c2444b5bdbc admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:13.452994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1022/2038] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:59:13.460326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ca59b-ae82-4beb-be37-6c2444b5bdbc admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:13.463574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-503ca59b-ae82-4beb-be37-6c2444b5bdbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:13.485871 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ca59b-ae82-4beb-be37-6c2444b5bdbc admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:13.492033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8658a25-abe0-4660-895a-a14b14d09c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:13.492985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8658a25-abe0-4660-895a-a14b14d09c7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:13.493217 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8658a25-abe0-4660-895a-a14b14d09c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:13.493535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8658a25-abe0-4660-895a-a14b14d09c7f admin admin] PATH_INFO: `/v3/projects/be988d2808974611b7afdf7b79eb31da/users/8db7a3227d3141229a1132bb2262334a/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:13.496706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8658a25-abe0-4660-895a-a14b14d09c7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=be988d2808974611b7afdf7b79eb31da, user_id=8db7a3227d3141229a1132bb2262334a, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:13.501705 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f2a97-b33a-4051-b735-edc13bbef483 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:13.503259 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8658a25-abe0-4660-895a-a14b14d09c7f admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:13.511371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea0f74-cdc9-415f-a5d0-6f4429e2b755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:13.513194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea0f74-cdc9-415f-a5d0-6f4429e2b755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:13.513790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea0f74-cdc9-415f-a5d0-6f4429e2b755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:13.514580 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea0f74-cdc9-415f-a5d0-6f4429e2b755 admin admin] PATH_INFO: `/v3/projects/e83d3e12fbb944ee867471e1e4ada2f3` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:13.516259 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea0f74-cdc9-415f-a5d0-6f4429e2b755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e83d3e12fbb944ee867471e1e4ada2f3)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:13.525763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1023/2039] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:13 2022] PUT /identity/v3/projects/be988d2808974611b7afdf7b79eb31da/users/8db7a3227d3141229a1132bb2262334a/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:13.525915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea0f74-cdc9-415f-a5d0-6f4429e2b755 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:13.529159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a208a5c-cbbc-4ae0-b973-886cf7d89220 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:13.529451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a208a5c-cbbc-4ae0-b973-886cf7d89220 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:13.530045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a208a5c-cbbc-4ae0-b973-886cf7d89220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:13.573224 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-9a208a5c-cbbc-4ae0-b973-886cf7d89220 None None] MFA Rules not processed for user `8db7a3227d3141229a1132bb2262334a`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:13.574965 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-9a208a5c-cbbc-4ae0-b973-886cf7d89220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:13.590558 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-69ea0f74-cdc9-415f-a5d0-6f4429e2b755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e83d3e12fbb944ee867471e1e4ada2f3', 'request_id': 'req-69ea0f74-cdc9-415f-a5d0-6f4429e2b755'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:13.604505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-69ea0f74-cdc9-415f-a5d0-6f4429e2b755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e83d3e12fbb944ee867471e1e4ada2f3', 'request_id': 'req-69ea0f74-cdc9-415f-a5d0-6f4429e2b755'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:13.620964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-69ea0f74-cdc9-415f-a5d0-6f4429e2b755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e83d3e12fbb944ee867471e1e4ada2f3', 'request_id': 'req-69ea0f74-cdc9-415f-a5d0-6f4429e2b755'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:13.625958 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:13.644421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-69ea0f74-cdc9-415f-a5d0-6f4429e2b755 admin admin] The token cache is being invalidate because project e83d3e12fbb944ee867471e1e4ada2f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:13.646127 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-69ea0f74-cdc9-415f-a5d0-6f4429e2b755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:13.647027 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-69ea0f74-cdc9-415f-a5d0-6f4429e2b755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:13.653086 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1017/2040] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:13 2022] DELETE /identity/v3/projects/e83d3e12fbb944ee867471e1e4ada2f3 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:13.654709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59321b70-d99d-4dda-b78c-b0b795c19703 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:13.655203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59321b70-d99d-4dda-b78c-b0b795c19703 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:13.655484 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59321b70-d99d-4dda-b78c-b0b795c19703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:13.683024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-59321b70-d99d-4dda-b78c-b0b795c19703 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:13.684268 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-59321b70-d99d-4dda-b78c-b0b795c19703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:13.711688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1024/2041] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:13.714262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a208a5c-cbbc-4ae0-b973-886cf7d89220 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:13.716832 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-9a208a5c-cbbc-4ae0-b973-886cf7d89220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:13.752279 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:13.780951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1018/2042] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:13.788616 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91f905-eab2-45bd-8be8-caa7a810547f None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:13.789277 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91f905-eab2-45bd-8be8-caa7a810547f None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:13.789504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91f905-eab2-45bd-8be8-caa7a810547f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:13.808252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a208a5c-cbbc-4ae0-b973-886cf7d89220 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:13.813472 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44055dd1-4947-4876-b952-fd0be7a55045 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:13.814108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44055dd1-4947-4876-b952-fd0be7a55045 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:13.814391 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44055dd1-4947-4876-b952-fd0be7a55045 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:13.814761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44055dd1-4947-4876-b952-fd0be7a55045 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:13.816229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-cb91f905-eab2-45bd-8be8-caa7a810547f None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:13.817050 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-44055dd1-4947-4876-b952-fd0be7a55045 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:13.817399 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-cb91f905-eab2-45bd-8be8-caa7a810547f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:13.826610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1019/2043] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:13.827453 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91f905-eab2-45bd-8be8-caa7a810547f None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:13.828849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-cb91f905-eab2-45bd-8be8-caa7a810547f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:13.843443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91f905-eab2-45bd-8be8-caa7a810547f None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:13.848266 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee1b8f-b593-48e0-bfe8-e31ce07f9f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:13.848945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee1b8f-b593-48e0-bfe8-e31ce07f9f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:13.849224 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee1b8f-b593-48e0-bfe8-e31ce07f9f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:13.849500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee1b8f-b593-48e0-bfe8-e31ce07f9f0d admin admin] PATH_INFO: `/v3/projects/ae0353ce58a240689a6b365b391cb550` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:13.850626 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee1b8f-b593-48e0-bfe8-e31ce07f9f0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae0353ce58a240689a6b365b391cb550)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:13.852762 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee1b8f-b593-48e0-bfe8-e31ce07f9f0d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:13.909562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-18ee1b8f-b593-48e0-bfe8-e31ce07f9f0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae0353ce58a240689a6b365b391cb550', 'request_id': 'req-18ee1b8f-b593-48e0-bfe8-e31ce07f9f0d'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:13.910788 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44055dd1-4947-4876-b952-fd0be7a55045 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:13.913588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44055dd1-4947-4876-b952-fd0be7a55045 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:13.916412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-18ee1b8f-b593-48e0-bfe8-e31ce07f9f0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae0353ce58a240689a6b365b391cb550', 'request_id': 'req-18ee1b8f-b593-48e0-bfe8-e31ce07f9f0d'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:13.923048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-18ee1b8f-b593-48e0-bfe8-e31ce07f9f0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae0353ce58a240689a6b365b391cb550', 'request_id': 'req-18ee1b8f-b593-48e0-bfe8-e31ce07f9f0d'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:13.924916 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1025/2044] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:13.927652 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44055dd1-4947-4876-b952-fd0be7a55045 service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:13.938257 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44055dd1-4947-4876-b952-fd0be7a55045 service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:13.952711 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-18ee1b8f-b593-48e0-bfe8-e31ce07f9f0d admin admin] The token cache is being invalidate because project ae0353ce58a240689a6b365b391cb550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:13.953153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-18ee1b8f-b593-48e0-bfe8-e31ce07f9f0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:13.953849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-18ee1b8f-b593-48e0-bfe8-e31ce07f9f0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:13.956519 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1020/2045] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:13 2022] DELETE /identity/v3/projects/ae0353ce58a240689a6b365b391cb550 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:13.962332 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640cba48-cfb9-478f-bc9d-1ac77443c8ca None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:13.963322 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640cba48-cfb9-478f-bc9d-1ac77443c8ca None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:13.963604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640cba48-cfb9-478f-bc9d-1ac77443c8ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:13.996541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-640cba48-cfb9-478f-bc9d-1ac77443c8ca None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:13.997983 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-640cba48-cfb9-478f-bc9d-1ac77443c8ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:14.012799 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc28797-425b-4aa2-8ab5-5a5e00b02be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:14.013427 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc28797-425b-4aa2-8ab5-5a5e00b02be7 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:14.013709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc28797-425b-4aa2-8ab5-5a5e00b02be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:14.013985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc28797-425b-4aa2-8ab5-5a5e00b02be7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:14.014653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc28797-425b-4aa2-8ab5-5a5e00b02be7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:14.014984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc28797-425b-4aa2-8ab5-5a5e00b02be7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:14.016681 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc28797-425b-4aa2-8ab5-5a5e00b02be7 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:14.033904 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1026/2046] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:14.038026 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc28797-425b-4aa2-8ab5-5a5e00b02be7 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:14.042061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-adc28797-425b-4aa2-8ab5-5a5e00b02be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:14.063585 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc28797-425b-4aa2-8ab5-5a5e00b02be7 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:14.070597 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9602b3-a019-4876-bb82-85095603bf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:14.071123 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9602b3-a019-4876-bb82-85095603bf64 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:14.071403 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9602b3-a019-4876-bb82-85095603bf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:14.072253 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9602b3-a019-4876-bb82-85095603bf64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:14.073184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9602b3-a019-4876-bb82-85095603bf64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:14.075660 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9602b3-a019-4876-bb82-85095603bf64 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:14.079253 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1021/2047] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:14.089063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8081c-897d-4592-a434-a59118e29317 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:14.089409 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8081c-897d-4592-a434-a59118e29317 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:14.089891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8081c-897d-4592-a434-a59118e29317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:14.122841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1027/2048] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:14 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:14.126990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9602b3-a019-4876-bb82-85095603bf64 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:14.136887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-30b8081c-897d-4592-a434-a59118e29317 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:14.138305 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-30b8081c-897d-4592-a434-a59118e29317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:14.148925 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:14.208311 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:14.240443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9602b3-a019-4876-bb82-85095603bf64 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:14.258225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1022/2049] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:14.259328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae3c15-e499-4ad9-8c29-862f74435493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:14.260473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae3c15-e499-4ad9-8c29-862f74435493 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:14.260753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae3c15-e499-4ad9-8c29-862f74435493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:14.261136 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae3c15-e499-4ad9-8c29-862f74435493 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:14.261666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae3c15-e499-4ad9-8c29-862f74435493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:14.263730 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae3c15-e499-4ad9-8c29-862f74435493 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:14.264956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8081c-897d-4592-a434-a59118e29317 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:14.298797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1028/2050] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:14 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:14.299356 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8081c-897d-4592-a434-a59118e29317 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:14.300218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae3c15-e499-4ad9-8c29-862f74435493 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:14.308449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bffa6-2800-4985-8afe-d4730896b37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:14.309201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bffa6-2800-4985-8afe-d4730896b37a admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:14.309623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bffa6-2800-4985-8afe-d4730896b37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:14.309946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bffa6-2800-4985-8afe-d4730896b37a admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:14.311475 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53bffa6-2800-4985-8afe-d4730896b37a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:14.311787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53bffa6-2800-4985-8afe-d4730896b37a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:14.314184 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53bffa6-2800-4985-8afe-d4730896b37a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:14.345426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae3c15-e499-4ad9-8c29-862f74435493 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:14.360907 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd022e7-ac24-4017-8dbe-666f60366904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:14.361775 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd022e7-ac24-4017-8dbe-666f60366904 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:14.362476 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd022e7-ac24-4017-8dbe-666f60366904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:14.362968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd022e7-ac24-4017-8dbe-666f60366904 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:14.363807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd022e7-ac24-4017-8dbe-666f60366904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:14.365945 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd022e7-ac24-4017-8dbe-666f60366904 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:14.369426 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1023/2051] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:14.370472 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bffa6-2800-4985-8afe-d4730896b37a admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:14.383186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bffa6-2800-4985-8afe-d4730896b37a admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:14.389443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdef31-40bc-4fbd-84b7-f5f4be159698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:14.390160 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdef31-40bc-4fbd-84b7-f5f4be159698 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:14.390352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdef31-40bc-4fbd-84b7-f5f4be159698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:14.390697 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdef31-40bc-4fbd-84b7-f5f4be159698 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:14.391371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fdef31-40bc-4fbd-84b7-f5f4be159698 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:14.393848 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fdef31-40bc-4fbd-84b7-f5f4be159698 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:14.407382 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1029/2052] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:14.409579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd022e7-ac24-4017-8dbe-666f60366904 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:14.426851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd022e7-ac24-4017-8dbe-666f60366904 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:14.432064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305410e-1a49-4cc9-84f5-6ad65f7e1898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:14.432700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305410e-1a49-4cc9-84f5-6ad65f7e1898 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:14.432976 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305410e-1a49-4cc9-84f5-6ad65f7e1898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:14.433246 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305410e-1a49-4cc9-84f5-6ad65f7e1898 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:14.433605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1024/2053] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:14 2022] POST /identity/v3/projects => generated 361 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:14.434879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305410e-1a49-4cc9-84f5-6ad65f7e1898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:14.435998 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305410e-1a49-4cc9-84f5-6ad65f7e1898 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:14.438217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdef31-40bc-4fbd-84b7-f5f4be159698 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:14.457063 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1030/2054] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:14.461831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305410e-1a49-4cc9-84f5-6ad65f7e1898 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:14.477036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305410e-1a49-4cc9-84f5-6ad65f7e1898 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:14.531747 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdef31-40bc-4fbd-84b7-f5f4be159698 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:14.537164 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f83330-3fcb-4c62-ab38-8c1d1af6b3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:14.537719 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f83330-3fcb-4c62-ab38-8c1d1af6b3f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:14.538059 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f83330-3fcb-4c62-ab38-8c1d1af6b3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:14.538319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f83330-3fcb-4c62-ab38-8c1d1af6b3f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:14.539008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f83330-3fcb-4c62-ab38-8c1d1af6b3f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:14.541174 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f83330-3fcb-4c62-ab38-8c1d1af6b3f4 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:14.553993 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adac9d1-0fbf-44d1-8fa4-485880e2cf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:14.557521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adac9d1-0fbf-44d1-8fa4-485880e2cf08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:14.558580 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adac9d1-0fbf-44d1-8fa4-485880e2cf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:14.559565 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adac9d1-0fbf-44d1-8fa4-485880e2cf08 admin admin] PATH_INFO: `/v3/projects/9d525d62168944648c5d72cb2cc4f8e2/users/79184aa0a79a4d85b3674541b43332d0/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:14.574690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1025/2055] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:14 2022] POST /identity/v3/users => generated 281 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:14.575946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f83330-3fcb-4c62-ab38-8c1d1af6b3f4 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:14.589413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f83330-3fcb-4c62-ab38-8c1d1af6b3f4 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:14.601117 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d138bc-cc42-49a2-adb3-0d085fd90bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:14.602056 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d138bc-cc42-49a2-adb3-0d085fd90bac admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:14.602056 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d138bc-cc42-49a2-adb3-0d085fd90bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:14.602478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d138bc-cc42-49a2-adb3-0d085fd90bac admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:14.603881 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d138bc-cc42-49a2-adb3-0d085fd90bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:14.609307 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d138bc-cc42-49a2-adb3-0d085fd90bac admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:14.622140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adac9d1-0fbf-44d1-8fa4-485880e2cf08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d525d62168944648c5d72cb2cc4f8e2, user_id=79184aa0a79a4d85b3674541b43332d0, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:14.622140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adac9d1-0fbf-44d1-8fa4-485880e2cf08 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:14.640520 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1026/2056] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:14.645266 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d138bc-cc42-49a2-adb3-0d085fd90bac admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:14.658337 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1031/2057] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:14 2022] PUT /identity/v3/projects/9d525d62168944648c5d72cb2cc4f8e2/users/79184aa0a79a4d85b3674541b43332d0/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:14.662938 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d138bc-cc42-49a2-adb3-0d085fd90bac admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:14.671453 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06810d-788c-47e5-9df7-90a65130d561 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:14.672022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06810d-788c-47e5-9df7-90a65130d561 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:14.673275 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06810d-788c-47e5-9df7-90a65130d561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:14.710722 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-5f06810d-788c-47e5-9df7-90a65130d561 None None] MFA Rules not processed for user `79184aa0a79a4d85b3674541b43332d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:14.712051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-5f06810d-788c-47e5-9df7-90a65130d561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:14.751020 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e948c4-bad7-4671-985f-5de1567aa660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:14.752122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e948c4-bad7-4671-985f-5de1567aa660 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:14.752367 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:14.753253 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e948c4-bad7-4671-985f-5de1567aa660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:14.753800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e948c4-bad7-4671-985f-5de1567aa660 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:14.754574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e948c4-bad7-4671-985f-5de1567aa660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:14.756588 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e948c4-bad7-4671-985f-5de1567aa660 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:14.767941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1027/2058] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:14.772356 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e948c4-bad7-4671-985f-5de1567aa660 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:14.782853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e948c4-bad7-4671-985f-5de1567aa660 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:14.788114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb1017-3de0-4533-a249-9d67046d2f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:14.788804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb1017-3de0-4533-a249-9d67046d2f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:14.789158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb1017-3de0-4533-a249-9d67046d2f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:14.789538 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb1017-3de0-4533-a249-9d67046d2f89 admin admin] PATH_INFO: `/v3/projects/1ebcf623ffc84e418ed8a0bc1a173c3d/users/bb6f985180b8449cb3ac4e0efa3f2f88/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:14.794771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1032/2059] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:14.804936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06810d-788c-47e5-9df7-90a65130d561 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:14.806978 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-5f06810d-788c-47e5-9df7-90a65130d561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:14.807284 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb1017-3de0-4533-a249-9d67046d2f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ebcf623ffc84e418ed8a0bc1a173c3d, user_id=bb6f985180b8449cb3ac4e0efa3f2f88, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:14.809946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb1017-3de0-4533-a249-9d67046d2f89 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:14.825202 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1028/2060] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:14 2022] PUT /identity/v3/projects/1ebcf623ffc84e418ed8a0bc1a173c3d/users/bb6f985180b8449cb3ac4e0efa3f2f88/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:14.837390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c14a9d-e984-4a3c-9c49-8b8bc5297237 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:14.837664 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c14a9d-e984-4a3c-9c49-8b8bc5297237 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:14.837935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c14a9d-e984-4a3c-9c49-8b8bc5297237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:14.845406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:14.873380 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-f6c14a9d-e984-4a3c-9c49-8b8bc5297237 None None] MFA Rules not processed for user `bb6f985180b8449cb3ac4e0efa3f2f88`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:14.874606 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f6c14a9d-e984-4a3c-9c49-8b8bc5297237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:14.907782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06810d-788c-47e5-9df7-90a65130d561 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:14.955669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1029/2061] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:14.972933 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e245edc-4834-48f7-b1e9-3086309eedee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5248cb1c39412680cce0d9891fe1ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:14.973930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e245edc-4834-48f7-b1e9-3086309eedee service glance] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:14.975284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e245edc-4834-48f7-b1e9-3086309eedee service glance] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:14.975284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e245edc-4834-48f7-b1e9-3086309eedee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:14.997139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c14a9d-e984-4a3c-9c49-8b8bc5297237 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:14.998657 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f6c14a9d-e984-4a3c-9c49-8b8bc5297237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:15.065296 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e245edc-4834-48f7-b1e9-3086309eedee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:15.067330 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e245edc-4834-48f7-b1e9-3086309eedee service glance] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:15.069944 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c14a9d-e984-4a3c-9c49-8b8bc5297237 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:15.074451 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db76a6-3a17-426b-ab65-4d6cbe442dc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:15.075210 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db76a6-3a17-426b-ab65-4d6cbe442dc4 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:15.075556 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db76a6-3a17-426b-ab65-4d6cbe442dc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:15.075919 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db76a6-3a17-426b-ab65-4d6cbe442dc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:15.078279 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1033/2062] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:15.086114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db76a6-3a17-426b-ab65-4d6cbe442dc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:15.088195 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db76a6-3a17-426b-ab65-4d6cbe442dc4 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:15.099143 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1030/2063] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:59:15.119496 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e245edc-4834-48f7-b1e9-3086309eedee service glance] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:15.197866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e245edc-4834-48f7-b1e9-3086309eedee service glance] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:15.202822 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88822bea-5733-44a5-bbfa-ebb061da96e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:15.203470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88822bea-5733-44a5-bbfa-ebb061da96e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:15.203753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88822bea-5733-44a5-bbfa-ebb061da96e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:15.204034 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88822bea-5733-44a5-bbfa-ebb061da96e3 admin admin] PATH_INFO: `/v3/users/bb6f985180b8449cb3ac4e0efa3f2f88` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:15.205199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88822bea-5733-44a5-bbfa-ebb061da96e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb6f985180b8449cb3ac4e0efa3f2f88)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:15.207317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88822bea-5733-44a5-bbfa-ebb061da96e3 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:15.235406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:15.274328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-88822bea-5733-44a5-bbfa-ebb061da96e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb6f985180b8449cb3ac4e0efa3f2f88'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:15.285277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-88822bea-5733-44a5-bbfa-ebb061da96e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb6f985180b8449cb3ac4e0efa3f2f88'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:15.285893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-88822bea-5733-44a5-bbfa-ebb061da96e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb6f985180b8449cb3ac4e0efa3f2f88'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:15.303666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-88822bea-5733-44a5-bbfa-ebb061da96e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb6f985180b8449cb3ac4e0efa3f2f88'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:15.304559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-88822bea-5733-44a5-bbfa-ebb061da96e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb6f985180b8449cb3ac4e0efa3f2f88'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:15.322698 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1034/2064] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:15 2022] DELETE /identity/v3/users/bb6f985180b8449cb3ac4e0efa3f2f88 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:15.327530 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c385b9-6d68-4b2d-b220-44e62054fca7 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:15.327821 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c385b9-6d68-4b2d-b220-44e62054fca7 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:15.328126 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c385b9-6d68-4b2d-b220-44e62054fca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:15.368933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-b4c385b9-6d68-4b2d-b220-44e62054fca7 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:15.370472 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b4c385b9-6d68-4b2d-b220-44e62054fca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:15.457922 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1031/2065] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:15.476553 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88822bea-5733-44a5-bbfa-ebb061da96e3 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:15.478781 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-88822bea-5733-44a5-bbfa-ebb061da96e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:15.543189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88822bea-5733-44a5-bbfa-ebb061da96e3 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:15.547502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c98ae4-d760-4bd9-8d8c-3e5f62b18d85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:15.548367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c98ae4-d760-4bd9-8d8c-3e5f62b18d85 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:15.548701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c98ae4-d760-4bd9-8d8c-3e5f62b18d85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:15.549074 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c98ae4-d760-4bd9-8d8c-3e5f62b18d85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:15.560360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c98ae4-d760-4bd9-8d8c-3e5f62b18d85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:15.562407 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c98ae4-d760-4bd9-8d8c-3e5f62b18d85 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:15.575354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1035/2066] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:16.183299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c385b9-6d68-4b2d-b220-44e62054fca7 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:16.185752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b4c385b9-6d68-4b2d-b220-44e62054fca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:16.192614 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:16.314391 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c385b9-6d68-4b2d-b220-44e62054fca7 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:16.323484 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4436fa9-ab50-4c2a-91f4-ead1baf8305f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:16.324211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4436fa9-ab50-4c2a-91f4-ead1baf8305f service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:16.324481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4436fa9-ab50-4c2a-91f4-ead1baf8305f service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:16.324778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4436fa9-ab50-4c2a-91f4-ead1baf8305f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:16.336054 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a4436fa9-ab50-4c2a-91f4-ead1baf8305f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:16.341688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c98ae4-d760-4bd9-8d8c-3e5f62b18d85 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:16.345103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d2c98ae4-d760-4bd9-8d8c-3e5f62b18d85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:16.367414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c98ae4-d760-4bd9-8d8c-3e5f62b18d85 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:16.376764 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4186a-5d20-4df2-b063-a054ed728ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:16.377423 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4186a-5d20-4df2-b063-a054ed728ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:16.377703 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4186a-5d20-4df2-b063-a054ed728ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:16.377986 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4186a-5d20-4df2-b063-a054ed728ffe admin admin] PATH_INFO: `/v3/projects/1ebcf623ffc84e418ed8a0bc1a173c3d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:16.395659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4186a-5d20-4df2-b063-a054ed728ffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ebcf623ffc84e418ed8a0bc1a173c3d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:16.395659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4186a-5d20-4df2-b063-a054ed728ffe admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:16.447191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4436fa9-ab50-4c2a-91f4-ead1baf8305f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:16.455143 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4436fa9-ab50-4c2a-91f4-ead1baf8305f service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:16.459541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ffd4186a-5d20-4df2-b063-a054ed728ffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ebcf623ffc84e418ed8a0bc1a173c3d', 'request_id': 'req-ffd4186a-5d20-4df2-b063-a054ed728ffe'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:16.466368 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1032/2067] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:16.475015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ffd4186a-5d20-4df2-b063-a054ed728ffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ebcf623ffc84e418ed8a0bc1a173c3d', 'request_id': 'req-ffd4186a-5d20-4df2-b063-a054ed728ffe'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:16.482706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ffd4186a-5d20-4df2-b063-a054ed728ffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ebcf623ffc84e418ed8a0bc1a173c3d', 'request_id': 'req-ffd4186a-5d20-4df2-b063-a054ed728ffe'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:16.496697 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ffd4186a-5d20-4df2-b063-a054ed728ffe admin admin] The token cache is being invalidate because project 1ebcf623ffc84e418ed8a0bc1a173c3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:16.496986 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ffd4186a-5d20-4df2-b063-a054ed728ffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:16.513167 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ffd4186a-5d20-4df2-b063-a054ed728ffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:16.516413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1036/2068] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:16 2022] DELETE /identity/v3/projects/1ebcf623ffc84e418ed8a0bc1a173c3d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:16.521119 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc57a1a-2a4e-4aa8-942d-5759ef50b292 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:16.521524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc57a1a-2a4e-4aa8-942d-5759ef50b292 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:16.521651 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc57a1a-2a4e-4aa8-942d-5759ef50b292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:16.542929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-2dc57a1a-2a4e-4aa8-942d-5759ef50b292 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:16.544140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2dc57a1a-2a4e-4aa8-942d-5759ef50b292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:16.610902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1033/2069] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:16.619168 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9985af2-d4c7-487d-91f9-bfb0ae7aaa9b None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:16.619451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9985af2-d4c7-487d-91f9-bfb0ae7aaa9b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:16.619720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9985af2-d4c7-487d-91f9-bfb0ae7aaa9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:16.642297 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-b9985af2-d4c7-487d-91f9-bfb0ae7aaa9b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:16.643550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b9985af2-d4c7-487d-91f9-bfb0ae7aaa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:16.651671 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1037/2070] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:16.656050 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc57a1a-2a4e-4aa8-942d-5759ef50b292 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:16.665850 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc57a1a-2a4e-4aa8-942d-5759ef50b292 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:16.670626 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9d5b2-f842-489b-b409-da7a9a2a5f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:16.671306 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9d5b2-f842-489b-b409-da7a9a2a5f0f admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:16.671582 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9d5b2-f842-489b-b409-da7a9a2a5f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:16.671852 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9d5b2-f842-489b-b409-da7a9a2a5f0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:16.672622 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9d5b2-f842-489b-b409-da7a9a2a5f0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:16.672918 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9d5b2-f842-489b-b409-da7a9a2a5f0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:16.674506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9d5b2-f842-489b-b409-da7a9a2a5f0f admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:16.688200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1034/2071] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:16.692794 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9985af2-d4c7-487d-91f9-bfb0ae7aaa9b None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:16.704762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9985af2-d4c7-487d-91f9-bfb0ae7aaa9b None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:16.712782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38a821-d1c9-42dd-92d8-1aef87e11458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:16.713470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38a821-d1c9-42dd-92d8-1aef87e11458 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:16.713774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38a821-d1c9-42dd-92d8-1aef87e11458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:16.715408 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38a821-d1c9-42dd-92d8-1aef87e11458 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:16.716539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38a821-d1c9-42dd-92d8-1aef87e11458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:16.718999 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38a821-d1c9-42dd-92d8-1aef87e11458 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:16.745391 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:16.768615 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1038/2072] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:16 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:16.772835 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9d5b2-f842-489b-b409-da7a9a2a5f0f admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:16.785340 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9d5b2-f842-489b-b409-da7a9a2a5f0f admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:16.845327 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:16.868052 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba328c-6fbc-498d-bd08-381ae317e1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:16.870935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba328c-6fbc-498d-bd08-381ae317e1ba admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:16.870935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba328c-6fbc-498d-bd08-381ae317e1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:16.870935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba328c-6fbc-498d-bd08-381ae317e1ba admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:16.871409 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba328c-6fbc-498d-bd08-381ae317e1ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:16.873566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba328c-6fbc-498d-bd08-381ae317e1ba admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:16.905447 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1035/2073] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:16 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:16.909906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38a821-d1c9-42dd-92d8-1aef87e11458 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:16.926387 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38a821-d1c9-42dd-92d8-1aef87e11458 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:16.933667 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f2cbb-d7a9-4493-9665-43601852a1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:16.934560 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f2cbb-d7a9-4493-9665-43601852a1c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:16.934974 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f2cbb-d7a9-4493-9665-43601852a1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:16.935488 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f2cbb-d7a9-4493-9665-43601852a1c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:16.936527 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f2cbb-d7a9-4493-9665-43601852a1c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:16.938486 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f2cbb-d7a9-4493-9665-43601852a1c6 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:16.954706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1039/2074] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:59:16.956068 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba328c-6fbc-498d-bd08-381ae317e1ba admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:16.969945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba328c-6fbc-498d-bd08-381ae317e1ba admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:16.976490 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e975425-066c-450b-aaef-e0e6f2b95e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:16.977295 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e975425-066c-450b-aaef-e0e6f2b95e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:16.977568 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e975425-066c-450b-aaef-e0e6f2b95e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:16.978413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e975425-066c-450b-aaef-e0e6f2b95e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:16.979693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e975425-066c-450b-aaef-e0e6f2b95e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:16.980937 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e975425-066c-450b-aaef-e0e6f2b95e4f admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:16.997869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1036/2075] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:59:17.002132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f2cbb-d7a9-4493-9665-43601852a1c6 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:17.014932 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f2cbb-d7a9-4493-9665-43601852a1c6 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:17.022416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee273d40-a1f4-44cf-bced-d2becf95a64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:17.023465 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee273d40-a1f4-44cf-bced-d2becf95a64b admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:17.023955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee273d40-a1f4-44cf-bced-d2becf95a64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:17.024492 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee273d40-a1f4-44cf-bced-d2becf95a64b admin admin] PATH_INFO: `/v3/projects/495d28624eac48069fdff279e2f1b001/users/6465d1250d2146b2be8d17a70b8bb929/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:17.057413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee273d40-a1f4-44cf-bced-d2becf95a64b admin admin] RBAC: Authorizing `identity:create_grant(project_id=495d28624eac48069fdff279e2f1b001, user_id=6465d1250d2146b2be8d17a70b8bb929, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:17.062169 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee273d40-a1f4-44cf-bced-d2becf95a64b admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:17.095079 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1040/2076] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:17 2022] PUT /identity/v3/projects/495d28624eac48069fdff279e2f1b001/users/6465d1250d2146b2be8d17a70b8bb929/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:17.108730 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6768d02-50e9-403a-bd4a-0875927f8b2c None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:17.108730 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6768d02-50e9-403a-bd4a-0875927f8b2c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:17.108730 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6768d02-50e9-403a-bd4a-0875927f8b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:17.125038 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-f6768d02-50e9-403a-bd4a-0875927f8b2c None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:17.126217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f6768d02-50e9-403a-bd4a-0875927f8b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:17.222716 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1037/2077] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:17.252190 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee273d40-a1f4-44cf-bced-d2becf95a64b admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:17.254120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ee273d40-a1f4-44cf-bced-d2becf95a64b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:17.316749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6768d02-50e9-403a-bd4a-0875927f8b2c None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:17.318485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f6768d02-50e9-403a-bd4a-0875927f8b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:17.348259 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee273d40-a1f4-44cf-bced-d2becf95a64b admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:17.352215 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8a8f4-40e8-4a08-b2b9-8cbb2e9a28f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:17.352769 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8a8f4-40e8-4a08-b2b9-8cbb2e9a28f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:17.353621 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8a8f4-40e8-4a08-b2b9-8cbb2e9a28f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:17.353996 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8a8f4-40e8-4a08-b2b9-8cbb2e9a28f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:17.375833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8a8f4-40e8-4a08-b2b9-8cbb2e9a28f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:17.377423 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8a8f4-40e8-4a08-b2b9-8cbb2e9a28f2 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:17.391549 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1041/2078] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:17.397720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6768d02-50e9-403a-bd4a-0875927f8b2c None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:17.411227 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04aa1f-cabf-4c5f-94e2-3bb73a5e6114 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:17.412185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04aa1f-cabf-4c5f-94e2-3bb73a5e6114 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:17.412471 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04aa1f-cabf-4c5f-94e2-3bb73a5e6114 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:17.412870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04aa1f-cabf-4c5f-94e2-3bb73a5e6114 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:17.414966 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-ab04aa1f-cabf-4c5f-94e2-3bb73a5e6114 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:17.527018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04aa1f-cabf-4c5f-94e2-3bb73a5e6114 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:17.527018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04aa1f-cabf-4c5f-94e2-3bb73a5e6114 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:17.527763 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1038/2079] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:18.226754 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596ebd2-076c-4088-a5e6-8781dde55a5a None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:18.227073 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596ebd2-076c-4088-a5e6-8781dde55a5a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:18.227346 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596ebd2-076c-4088-a5e6-8781dde55a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:18.261071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-f596ebd2-076c-4088-a5e6-8781dde55a5a None None] MFA Rules not processed for user `6465d1250d2146b2be8d17a70b8bb929`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:18.262312 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-f596ebd2-076c-4088-a5e6-8781dde55a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:18.363723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1042/2080] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:18.375918 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04aa1f-cabf-4c5f-94e2-3bb73a5e6114 service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:18.386802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04aa1f-cabf-4c5f-94e2-3bb73a5e6114 service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:18.391033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385743cc-c924-4b99-bcb4-4a7134ed3ddd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:18.391726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385743cc-c924-4b99-bcb4-4a7134ed3ddd service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:18.392079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385743cc-c924-4b99-bcb4-4a7134ed3ddd service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:18.392436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385743cc-c924-4b99-bcb4-4a7134ed3ddd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:18.405702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385743cc-c924-4b99-bcb4-4a7134ed3ddd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:18.408291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385743cc-c924-4b99-bcb4-4a7134ed3ddd service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:18.418298 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:18.424317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1039/2081] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:19.921502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935dad2a-05d9-4210-9587-5cad376d945a None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:19.922451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935dad2a-05d9-4210-9587-5cad376d945a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:19.922895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935dad2a-05d9-4210-9587-5cad376d945a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:19.947620 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-935dad2a-05d9-4210-9587-5cad376d945a None None] MFA Rules not processed for user `6465d1250d2146b2be8d17a70b8bb929`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:19.948761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-935dad2a-05d9-4210-9587-5cad376d945a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:19.956015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1043/2082] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:20.370806 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385743cc-c924-4b99-bcb4-4a7134ed3ddd service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:20.372468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-385743cc-c924-4b99-bcb4-4a7134ed3ddd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:20.392854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385743cc-c924-4b99-bcb4-4a7134ed3ddd service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:20.398108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2e8e3-73fa-4290-8f49-25fb7176819b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:20.398663 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2e8e3-73fa-4290-8f49-25fb7176819b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:20.398976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2e8e3-73fa-4290-8f49-25fb7176819b admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:20.399521 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2e8e3-73fa-4290-8f49-25fb7176819b admin admin] PATH_INFO: `/v3/users/f0a8778bfd00425a8810213931d4f071` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:20.401258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c2e8e3-73fa-4290-8f49-25fb7176819b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0a8778bfd00425a8810213931d4f071)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:20.403237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c2e8e3-73fa-4290-8f49-25fb7176819b admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:20.476846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f5c2e8e3-73fa-4290-8f49-25fb7176819b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0a8778bfd00425a8810213931d4f071'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:20.490363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f5c2e8e3-73fa-4290-8f49-25fb7176819b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0a8778bfd00425a8810213931d4f071'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:20.500801 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f5c2e8e3-73fa-4290-8f49-25fb7176819b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0a8778bfd00425a8810213931d4f071'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:20.516266 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f5c2e8e3-73fa-4290-8f49-25fb7176819b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0a8778bfd00425a8810213931d4f071'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:20.517063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f5c2e8e3-73fa-4290-8f49-25fb7176819b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0a8778bfd00425a8810213931d4f071'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:20.523584 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1040/2083] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:20 2022] DELETE /identity/v3/users/f0a8778bfd00425a8810213931d4f071 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:20.528518 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ca254-837d-48ed-81eb-097d545af715 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:20.528806 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ca254-837d-48ed-81eb-097d545af715 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:20.529083 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ca254-837d-48ed-81eb-097d545af715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:20.563642 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-009ca254-837d-48ed-81eb-097d545af715 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:20.565555 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-009ca254-837d-48ed-81eb-097d545af715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:20.640514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:20.695956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1044/2084] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:20.716551 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2e8e3-73fa-4290-8f49-25fb7176819b admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:20.718374 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f5c2e8e3-73fa-4290-8f49-25fb7176819b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:20.800377 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2e8e3-73fa-4290-8f49-25fb7176819b admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:20.806834 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713b72a-6c0e-4b64-9cfb-7528429b2fa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:20.807706 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713b72a-6c0e-4b64-9cfb-7528429b2fa4 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:20.808168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713b72a-6c0e-4b64-9cfb-7528429b2fa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:20.808592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713b72a-6c0e-4b64-9cfb-7528429b2fa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:20.821941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713b72a-6c0e-4b64-9cfb-7528429b2fa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:20.823638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713b72a-6c0e-4b64-9cfb-7528429b2fa4 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:20.835614 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1041/2085] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:21.380198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ca254-837d-48ed-81eb-097d545af715 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:21.386347 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-009ca254-837d-48ed-81eb-097d545af715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:21.401956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ca254-837d-48ed-81eb-097d545af715 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:21.407223 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab0065-e951-42a7-aec1-8b21f44309e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:21.408083 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab0065-e951-42a7-aec1-8b21f44309e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:21.408435 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab0065-e951-42a7-aec1-8b21f44309e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:21.408778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab0065-e951-42a7-aec1-8b21f44309e2 admin admin] PATH_INFO: `/v3/projects/da5f7d9e8430413d8f48cbd2af089a90` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:21.410075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab0065-e951-42a7-aec1-8b21f44309e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da5f7d9e8430413d8f48cbd2af089a90)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:21.412630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab0065-e951-42a7-aec1-8b21f44309e2 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:21.464296 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-57ab0065-e951-42a7-aec1-8b21f44309e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da5f7d9e8430413d8f48cbd2af089a90', 'request_id': 'req-57ab0065-e951-42a7-aec1-8b21f44309e2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.472643 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-57ab0065-e951-42a7-aec1-8b21f44309e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da5f7d9e8430413d8f48cbd2af089a90', 'request_id': 'req-57ab0065-e951-42a7-aec1-8b21f44309e2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.483441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-57ab0065-e951-42a7-aec1-8b21f44309e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da5f7d9e8430413d8f48cbd2af089a90', 'request_id': 'req-57ab0065-e951-42a7-aec1-8b21f44309e2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.495964 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-57ab0065-e951-42a7-aec1-8b21f44309e2 admin admin] The token cache is being invalidate because project da5f7d9e8430413d8f48cbd2af089a90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:21.496274 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-57ab0065-e951-42a7-aec1-8b21f44309e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.496910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-57ab0065-e951-42a7-aec1-8b21f44309e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.498418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1045/2086] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:21 2022] DELETE /identity/v3/projects/da5f7d9e8430413d8f48cbd2af089a90 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:21.504587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2eb038-bbe8-4e69-a511-dcd53471cd11 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:21.505080 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2eb038-bbe8-4e69-a511-dcd53471cd11 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:21.505252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2eb038-bbe8-4e69-a511-dcd53471cd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:21.527093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-dc2eb038-bbe8-4e69-a511-dcd53471cd11 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:21.528329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-dc2eb038-bbe8-4e69-a511-dcd53471cd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:21.594676 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1042/2087] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:21.604018 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c54358-6412-4411-97e9-1de693e337bd None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:21.604313 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c54358-6412-4411-97e9-1de693e337bd None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:21.604571 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c54358-6412-4411-97e9-1de693e337bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:21.625883 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-87c54358-6412-4411-97e9-1de693e337bd None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:21.627142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-87c54358-6412-4411-97e9-1de693e337bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:21.636003 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1046/2088] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:21.641135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2eb038-bbe8-4e69-a511-dcd53471cd11 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:21.654429 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2eb038-bbe8-4e69-a511-dcd53471cd11 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:21.660054 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114d227-360b-4f81-a24c-b013cbabd9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:21.660691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114d227-360b-4f81-a24c-b013cbabd9ed admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:21.660971 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114d227-360b-4f81-a24c-b013cbabd9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:21.661244 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114d227-360b-4f81-a24c-b013cbabd9ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:21.661898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d114d227-360b-4f81-a24c-b013cbabd9ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:21.662201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d114d227-360b-4f81-a24c-b013cbabd9ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:21.663842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d114d227-360b-4f81-a24c-b013cbabd9ed admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:21.680533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1043/2089] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:21.682159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c54358-6412-4411-97e9-1de693e337bd None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:21.692107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c54358-6412-4411-97e9-1de693e337bd None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:21.697034 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8498fd4-6315-499f-ace5-a48f421a3f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:21.697627 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8498fd4-6315-499f-ace5-a48f421a3f54 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:21.697923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8498fd4-6315-499f-ace5-a48f421a3f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:21.698171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8498fd4-6315-499f-ace5-a48f421a3f54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:21.698826 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8498fd4-6315-499f-ace5-a48f421a3f54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:21.700696 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8498fd4-6315-499f-ace5-a48f421a3f54 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:21.735490 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1047/2090] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:21 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:21.741383 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114d227-360b-4f81-a24c-b013cbabd9ed admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:21.746623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:21.754114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114d227-360b-4f81-a24c-b013cbabd9ed admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:21.807946 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8498fd4-6315-499f-ace5-a48f421a3f54 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:21.809708 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d8498fd4-6315-499f-ace5-a48f421a3f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:21.833626 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f4eca-3a8c-4494-bcca-21d4c36467bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:21.834253 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f4eca-3a8c-4494-bcca-21d4c36467bb admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:21.834525 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f4eca-3a8c-4494-bcca-21d4c36467bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:21.834831 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f4eca-3a8c-4494-bcca-21d4c36467bb admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:21.835472 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f4eca-3a8c-4494-bcca-21d4c36467bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:21.837941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f4eca-3a8c-4494-bcca-21d4c36467bb admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:21.867613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1044/2091] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:21 2022] POST /identity/v3/users => generated 286 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:21.871816 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f4eca-3a8c-4494-bcca-21d4c36467bb admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:21.885889 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f4eca-3a8c-4494-bcca-21d4c36467bb admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:21.890791 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04447bb2-9963-4244-b827-c9031567fb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:21.892010 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04447bb2-9963-4244-b827-c9031567fb5f admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:21.892010 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04447bb2-9963-4244-b827-c9031567fb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:21.892184 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04447bb2-9963-4244-b827-c9031567fb5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:21.892757 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04447bb2-9963-4244-b827-c9031567fb5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:21.894379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04447bb2-9963-4244-b827-c9031567fb5f admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:21.905299 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8498fd4-6315-499f-ace5-a48f421a3f54 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:21.906216 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1045/2092] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:59:21.910358 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04447bb2-9963-4244-b827-c9031567fb5f admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:21.910751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc19c84-7152-4431-b017-4b3ed1d863bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:21.911486 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc19c84-7152-4431-b017-4b3ed1d863bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:21.911826 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc19c84-7152-4431-b017-4b3ed1d863bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:21.912168 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc19c84-7152-4431-b017-4b3ed1d863bf admin admin] PATH_INFO: `/v3/users/fa9b638bc1414e85ae4c4c30c95effc1` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:21.913374 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc19c84-7152-4431-b017-4b3ed1d863bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa9b638bc1414e85ae4c4c30c95effc1)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:21.915970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc19c84-7152-4431-b017-4b3ed1d863bf admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:21.920959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04447bb2-9963-4244-b827-c9031567fb5f admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:21.925667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d8aad-5314-435d-886e-86aa99890a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:21.926293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d8aad-5314-435d-886e-86aa99890a2b admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:21.926574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d8aad-5314-435d-886e-86aa99890a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:21.926857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d8aad-5314-435d-886e-86aa99890a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:21.927495 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d8aad-5314-435d-886e-86aa99890a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:21.929079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d8aad-5314-435d-886e-86aa99890a2b admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:21.939373 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1046/2093] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:21.943620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d8aad-5314-435d-886e-86aa99890a2b admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:21.953733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d8aad-5314-435d-886e-86aa99890a2b admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:21.959439 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d4732-34cd-422b-a880-4b09cb497b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:21.960219 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d4732-34cd-422b-a880-4b09cb497b81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:21.960526 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d4732-34cd-422b-a880-4b09cb497b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:21.960806 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d4732-34cd-422b-a880-4b09cb497b81 admin admin] PATH_INFO: `/v3/projects/fcbd57382c9d49dfad81e175ae6cfbb5/users/a201d7ceaa244a59a357637d7f4c754a/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:21.979213 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d4732-34cd-422b-a880-4b09cb497b81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcbd57382c9d49dfad81e175ae6cfbb5, user_id=a201d7ceaa244a59a357637d7f4c754a, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:21.981283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:21.982007 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d4732-34cd-422b-a880-4b09cb497b81 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:21.984806 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-8fc19c84-7152-4431-b017-4b3ed1d863bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa9b638bc1414e85ae4c4c30c95effc1'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.994321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-8fc19c84-7152-4431-b017-4b3ed1d863bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa9b638bc1414e85ae4c4c30c95effc1'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.995047 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-8fc19c84-7152-4431-b017-4b3ed1d863bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa9b638bc1414e85ae4c4c30c95effc1'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:21.996180 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1047/2094] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:21 2022] PUT /identity/v3/projects/fcbd57382c9d49dfad81e175ae6cfbb5/users/a201d7ceaa244a59a357637d7f4c754a/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:22.007776 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-8fc19c84-7152-4431-b017-4b3ed1d863bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa9b638bc1414e85ae4c4c30c95effc1'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.008377 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31191bc-7fbf-46e2-89df-3dbc7f3f0908 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:22.008560 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-8fc19c84-7152-4431-b017-4b3ed1d863bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa9b638bc1414e85ae4c4c30c95effc1'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.008747 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31191bc-7fbf-46e2-89df-3dbc7f3f0908 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:22.009059 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31191bc-7fbf-46e2-89df-3dbc7f3f0908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:22.027039 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1048/2095] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:21 2022] DELETE /identity/v3/users/fa9b638bc1414e85ae4c4c30c95effc1 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:22.028611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003f092-0c7c-4887-9c9e-2297b8eb9936 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:22.028944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003f092-0c7c-4887-9c9e-2297b8eb9936 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:22.029304 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003f092-0c7c-4887-9c9e-2297b8eb9936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:22.040887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-b31191bc-7fbf-46e2-89df-3dbc7f3f0908 None None] MFA Rules not processed for user `a201d7ceaa244a59a357637d7f4c754a`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:22.042680 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b31191bc-7fbf-46e2-89df-3dbc7f3f0908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:22.058226 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-7003f092-0c7c-4887-9c9e-2297b8eb9936 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:22.059557 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7003f092-0c7c-4887-9c9e-2297b8eb9936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:22.143088 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1049/2096] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:22.143279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1048/2097] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:22.150529 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003f092-0c7c-4887-9c9e-2297b8eb9936 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:22.152287 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7003f092-0c7c-4887-9c9e-2297b8eb9936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:22.157200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31191bc-7fbf-46e2-89df-3dbc7f3f0908 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:22.159214 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b31191bc-7fbf-46e2-89df-3dbc7f3f0908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:22.218276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:22.231714 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003f092-0c7c-4887-9c9e-2297b8eb9936 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:22.235991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ab408-b731-432e-8597-ceadba7eccf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:22.236683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ab408-b731-432e-8597-ceadba7eccf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:22.236994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ab408-b731-432e-8597-ceadba7eccf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:22.237237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ab408-b731-432e-8597-ceadba7eccf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:22.239870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31191bc-7fbf-46e2-89df-3dbc7f3f0908 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:22.245744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead01a8-c6ca-4e95-b191-4f9d1a4fc4e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:22.246372 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead01a8-c6ca-4e95-b191-4f9d1a4fc4e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:22.246642 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead01a8-c6ca-4e95-b191-4f9d1a4fc4e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:22.246939 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead01a8-c6ca-4e95-b191-4f9d1a4fc4e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:22.249057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ab408-b731-432e-8597-ceadba7eccf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:22.251091 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ab408-b731-432e-8597-ceadba7eccf0 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:22.256725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cead01a8-c6ca-4e95-b191-4f9d1a4fc4e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:22.263529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cead01a8-c6ca-4e95-b191-4f9d1a4fc4e6 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:22.269307 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1049/2098] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:59:22.285825 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1050/2099] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:22.598837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead01a8-c6ca-4e95-b191-4f9d1a4fc4e6 service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:22.602447 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-cead01a8-c6ca-4e95-b191-4f9d1a4fc4e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:22.621446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead01a8-c6ca-4e95-b191-4f9d1a4fc4e6 service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:22.626404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398096ab-b991-4882-b569-782ced8ca150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:22.627512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398096ab-b991-4882-b569-782ced8ca150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:22.627792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398096ab-b991-4882-b569-782ced8ca150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:22.628064 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398096ab-b991-4882-b569-782ced8ca150 admin admin] PATH_INFO: `/v3/projects/839309f2f3404355a43f2c0664445e44` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:22.629232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398096ab-b991-4882-b569-782ced8ca150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=839309f2f3404355a43f2c0664445e44)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:22.631331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398096ab-b991-4882-b569-782ced8ca150 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:22.698891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-398096ab-b991-4882-b569-782ced8ca150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '839309f2f3404355a43f2c0664445e44', 'request_id': 'req-398096ab-b991-4882-b569-782ced8ca150'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.705306 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-398096ab-b991-4882-b569-782ced8ca150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '839309f2f3404355a43f2c0664445e44', 'request_id': 'req-398096ab-b991-4882-b569-782ced8ca150'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.710955 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-398096ab-b991-4882-b569-782ced8ca150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '839309f2f3404355a43f2c0664445e44', 'request_id': 'req-398096ab-b991-4882-b569-782ced8ca150'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.729289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-398096ab-b991-4882-b569-782ced8ca150 admin admin] The token cache is being invalidate because project 839309f2f3404355a43f2c0664445e44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:22.729684 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-398096ab-b991-4882-b569-782ced8ca150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.730397 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-398096ab-b991-4882-b569-782ced8ca150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.732010 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1050/2100] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:22 2022] DELETE /identity/v3/projects/839309f2f3404355a43f2c0664445e44 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:22.736400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ab408-b731-432e-8597-ceadba7eccf0 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:22.747628 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ab408-b731-432e-8597-ceadba7eccf0 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:22.804115 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf372acf-b43f-4802-a1d0-58dad652adf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:22.804649 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf372acf-b43f-4802-a1d0-58dad652adf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:22.804925 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf372acf-b43f-4802-a1d0-58dad652adf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:22.805193 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf372acf-b43f-4802-a1d0-58dad652adf1 admin admin] PATH_INFO: `/v3/users/5eff3d4eadeb4a7f97805ce162aec3c2` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:22.806421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf372acf-b43f-4802-a1d0-58dad652adf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eff3d4eadeb4a7f97805ce162aec3c2)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:22.808370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf372acf-b43f-4802-a1d0-58dad652adf1 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:22.812974 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:22.861000 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bf372acf-b43f-4802-a1d0-58dad652adf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eff3d4eadeb4a7f97805ce162aec3c2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.869207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bf372acf-b43f-4802-a1d0-58dad652adf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eff3d4eadeb4a7f97805ce162aec3c2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.869909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bf372acf-b43f-4802-a1d0-58dad652adf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eff3d4eadeb4a7f97805ce162aec3c2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.887036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bf372acf-b43f-4802-a1d0-58dad652adf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eff3d4eadeb4a7f97805ce162aec3c2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.888111 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bf372acf-b43f-4802-a1d0-58dad652adf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eff3d4eadeb4a7f97805ce162aec3c2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:22.905382 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1051/2101] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:22 2022] DELETE /identity/v3/users/5eff3d4eadeb4a7f97805ce162aec3c2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:23.238691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398096ab-b991-4882-b569-782ced8ca150 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:23.240160 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-398096ab-b991-4882-b569-782ced8ca150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:23.261632 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398096ab-b991-4882-b569-782ced8ca150 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:23.336581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc20e5-5d34-4c12-bc08-b6b2493610e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:23.337333 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc20e5-5d34-4c12-bc08-b6b2493610e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:23.337609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc20e5-5d34-4c12-bc08-b6b2493610e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:23.337881 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc20e5-5d34-4c12-bc08-b6b2493610e0 admin admin] PATH_INFO: `/v3/projects/32acdf41c87240b998366ec8d6ca513c` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:23.339091 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc20e5-5d34-4c12-bc08-b6b2493610e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32acdf41c87240b998366ec8d6ca513c)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:23.341297 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc20e5-5d34-4c12-bc08-b6b2493610e0 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:23.410680 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-60dc20e5-5d34-4c12-bc08-b6b2493610e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32acdf41c87240b998366ec8d6ca513c', 'request_id': 'req-60dc20e5-5d34-4c12-bc08-b6b2493610e0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:23.417086 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-60dc20e5-5d34-4c12-bc08-b6b2493610e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32acdf41c87240b998366ec8d6ca513c', 'request_id': 'req-60dc20e5-5d34-4c12-bc08-b6b2493610e0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:23.422590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-60dc20e5-5d34-4c12-bc08-b6b2493610e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32acdf41c87240b998366ec8d6ca513c', 'request_id': 'req-60dc20e5-5d34-4c12-bc08-b6b2493610e0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:23.424269 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:23.437766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-60dc20e5-5d34-4c12-bc08-b6b2493610e0 admin admin] The token cache is being invalidate because project 32acdf41c87240b998366ec8d6ca513c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:23.438095 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-60dc20e5-5d34-4c12-bc08-b6b2493610e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:23.438725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-60dc20e5-5d34-4c12-bc08-b6b2493610e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:23.440434 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1051/2102] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:23 2022] DELETE /identity/v3/projects/32acdf41c87240b998366ec8d6ca513c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:23.445571 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf372acf-b43f-4802-a1d0-58dad652adf1 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:23.447708 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-bf372acf-b43f-4802-a1d0-58dad652adf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:23.467029 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf372acf-b43f-4802-a1d0-58dad652adf1 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:23.523976 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191fe8e-1675-47cf-8005-ae0db75d5a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:23.524643 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191fe8e-1675-47cf-8005-ae0db75d5a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:23.524921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191fe8e-1675-47cf-8005-ae0db75d5a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:23.525191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191fe8e-1675-47cf-8005-ae0db75d5a84 admin admin] PATH_INFO: `/v3/users/8db7a3227d3141229a1132bb2262334a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:23.526307 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191fe8e-1675-47cf-8005-ae0db75d5a84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8db7a3227d3141229a1132bb2262334a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:23.528674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191fe8e-1675-47cf-8005-ae0db75d5a84 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:23.585564 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9191fe8e-1675-47cf-8005-ae0db75d5a84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8db7a3227d3141229a1132bb2262334a'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:23.594307 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9191fe8e-1675-47cf-8005-ae0db75d5a84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8db7a3227d3141229a1132bb2262334a'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:23.595030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9191fe8e-1675-47cf-8005-ae0db75d5a84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8db7a3227d3141229a1132bb2262334a'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:23.607916 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9191fe8e-1675-47cf-8005-ae0db75d5a84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8db7a3227d3141229a1132bb2262334a'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:23.608570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9191fe8e-1675-47cf-8005-ae0db75d5a84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8db7a3227d3141229a1132bb2262334a'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:23.621491 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1052/2103] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:23 2022] DELETE /identity/v3/users/8db7a3227d3141229a1132bb2262334a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:23.925744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc20e5-5d34-4c12-bc08-b6b2493610e0 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:23.928067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-60dc20e5-5d34-4c12-bc08-b6b2493610e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:23.944913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc20e5-5d34-4c12-bc08-b6b2493610e0 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:24.021825 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a45303-f906-44d1-8721-faba742d80bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:24.023148 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a45303-f906-44d1-8721-faba742d80bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:24.023702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a45303-f906-44d1-8721-faba742d80bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:24.023978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a45303-f906-44d1-8721-faba742d80bc admin admin] PATH_INFO: `/v3/projects/be988d2808974611b7afdf7b79eb31da` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:24.025855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a45303-f906-44d1-8721-faba742d80bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=be988d2808974611b7afdf7b79eb31da)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:24.028749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a45303-f906-44d1-8721-faba742d80bc admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:24.134073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-61a45303-f906-44d1-8721-faba742d80bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be988d2808974611b7afdf7b79eb31da', 'request_id': 'req-61a45303-f906-44d1-8721-faba742d80bc'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:24.141548 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-61a45303-f906-44d1-8721-faba742d80bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be988d2808974611b7afdf7b79eb31da', 'request_id': 'req-61a45303-f906-44d1-8721-faba742d80bc'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:24.149086 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-61a45303-f906-44d1-8721-faba742d80bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be988d2808974611b7afdf7b79eb31da', 'request_id': 'req-61a45303-f906-44d1-8721-faba742d80bc'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:24.172862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-61a45303-f906-44d1-8721-faba742d80bc admin admin] The token cache is being invalidate because project be988d2808974611b7afdf7b79eb31da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:24.173158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-61a45303-f906-44d1-8721-faba742d80bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:24.179194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-61a45303-f906-44d1-8721-faba742d80bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:24.181087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1052/2104] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:23 2022] DELETE /identity/v3/projects/be988d2808974611b7afdf7b79eb31da => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:24.186891 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd0970-732a-4311-9d55-2c0c256bf5ce None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:24.188350 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd0970-732a-4311-9d55-2c0c256bf5ce None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:24.188851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd0970-732a-4311-9d55-2c0c256bf5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:24.224415 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:24.241873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-5bdd0970-732a-4311-9d55-2c0c256bf5ce None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:24.243585 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-5bdd0970-732a-4311-9d55-2c0c256bf5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:24.329263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1053/2105] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:24.331404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ab047-7202-402f-bdc2-7afe076541e3 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:24.331896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ab047-7202-402f-bdc2-7afe076541e3 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:24.332176 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ab047-7202-402f-bdc2-7afe076541e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:24.353401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-bf5ab047-7202-402f-bdc2-7afe076541e3 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:24.354619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-bf5ab047-7202-402f-bdc2-7afe076541e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:24.363208 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1053/2106] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:24.367506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd0970-732a-4311-9d55-2c0c256bf5ce None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:24.377731 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd0970-732a-4311-9d55-2c0c256bf5ce None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:24.386576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b2256-f65e-4a2c-9961-1000da8c28d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:24.387281 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b2256-f65e-4a2c-9961-1000da8c28d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:24.387503 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b2256-f65e-4a2c-9961-1000da8c28d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:24.387770 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b2256-f65e-4a2c-9961-1000da8c28d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:24.388604 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b2256-f65e-4a2c-9961-1000da8c28d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:24.388801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b2256-f65e-4a2c-9961-1000da8c28d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:24.390463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b2256-f65e-4a2c-9961-1000da8c28d0 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:24.405716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1054/2107] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:24.408837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ab047-7202-402f-bdc2-7afe076541e3 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:24.418862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ab047-7202-402f-bdc2-7afe076541e3 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:24.423669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51666b6f-da37-4775-ac45-6408b19c9c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:24.424276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51666b6f-da37-4775-ac45-6408b19c9c79 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:24.424551 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51666b6f-da37-4775-ac45-6408b19c9c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:24.424823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51666b6f-da37-4775-ac45-6408b19c9c79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:24.425459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51666b6f-da37-4775-ac45-6408b19c9c79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:24.427584 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51666b6f-da37-4775-ac45-6408b19c9c79 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:24.469855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1054/2108] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:24 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:24.474112 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b2256-f65e-4a2c-9961-1000da8c28d0 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:24.485254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b2256-f65e-4a2c-9961-1000da8c28d0 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:24.572806 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0971c7-5d10-43a4-a6cb-75dcd46d2aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:24.573524 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0971c7-5d10-43a4-a6cb-75dcd46d2aed admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:24.574077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0971c7-5d10-43a4-a6cb-75dcd46d2aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:24.574401 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0971c7-5d10-43a4-a6cb-75dcd46d2aed admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:24.575117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0971c7-5d10-43a4-a6cb-75dcd46d2aed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:24.577256 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0971c7-5d10-43a4-a6cb-75dcd46d2aed admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:24.610600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1055/2109] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:24 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:24.614481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51666b6f-da37-4775-ac45-6408b19c9c79 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:24.627329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51666b6f-da37-4775-ac45-6408b19c9c79 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:24.638083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0146ae-b545-46ec-8fab-a8ac2aefcef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:24.638585 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0146ae-b545-46ec-8fab-a8ac2aefcef1 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:24.639033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0146ae-b545-46ec-8fab-a8ac2aefcef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:24.639398 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0146ae-b545-46ec-8fab-a8ac2aefcef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:24.640294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0146ae-b545-46ec-8fab-a8ac2aefcef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:24.642024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0146ae-b545-46ec-8fab-a8ac2aefcef1 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:24.662394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1055/2110] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:24.664288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0971c7-5d10-43a4-a6cb-75dcd46d2aed admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:24.676819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0971c7-5d10-43a4-a6cb-75dcd46d2aed admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:24.682857 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914484b5-37a8-4851-8502-d3fe88f52b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:24.683526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914484b5-37a8-4851-8502-d3fe88f52b42 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:24.683841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914484b5-37a8-4851-8502-d3fe88f52b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:24.684769 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914484b5-37a8-4851-8502-d3fe88f52b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:24.684769 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914484b5-37a8-4851-8502-d3fe88f52b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:24.686374 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914484b5-37a8-4851-8502-d3fe88f52b42 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:24.700504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1056/2111] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:59:24.704814 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0146ae-b545-46ec-8fab-a8ac2aefcef1 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:24.718725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0146ae-b545-46ec-8fab-a8ac2aefcef1 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:24.725705 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d7b41-7a80-4db1-9dfa-4b7d368f6c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:24.726358 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d7b41-7a80-4db1-9dfa-4b7d368f6c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:24.726652 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d7b41-7a80-4db1-9dfa-4b7d368f6c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:24.726939 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d7b41-7a80-4db1-9dfa-4b7d368f6c84 admin admin] PATH_INFO: `/v3/projects/1747e74c003b4fbabef2d6d827e2a555/users/63e15dd9ed344b74acfeb206bfe690f2/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:24.746345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d7b41-7a80-4db1-9dfa-4b7d368f6c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1747e74c003b4fbabef2d6d827e2a555, user_id=63e15dd9ed344b74acfeb206bfe690f2, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:24.749501 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d7b41-7a80-4db1-9dfa-4b7d368f6c84 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:24.778789 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1056/2112] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:24 2022] PUT /identity/v3/projects/1747e74c003b4fbabef2d6d827e2a555/users/63e15dd9ed344b74acfeb206bfe690f2/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:24.785493 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb142d4-6d82-46c1-bc89-f4cca083dd66 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:24.785784 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb142d4-6d82-46c1-bc89-f4cca083dd66 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:24.786061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb142d4-6d82-46c1-bc89-f4cca083dd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:24.818057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-7eb142d4-6d82-46c1-bc89-f4cca083dd66 None None] MFA Rules not processed for user `63e15dd9ed344b74acfeb206bfe690f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:24.819382 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7eb142d4-6d82-46c1-bc89-f4cca083dd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:24.912981 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:24.923070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1057/2113] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:24.932446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d7b41-7a80-4db1-9dfa-4b7d368f6c84 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:24.934241 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-197d7b41-7a80-4db1-9dfa-4b7d368f6c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:24.958597 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:25.006225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d7b41-7a80-4db1-9dfa-4b7d368f6c84 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:25.010890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e4042-64dc-4b4f-898e-d42c679c03e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:25.011633 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4042-64dc-4b4f-898e-d42c679c03e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:25.011978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4042-64dc-4b4f-898e-d42c679c03e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:25.012308 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4042-64dc-4b4f-898e-d42c679c03e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:25.021661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e4042-64dc-4b4f-898e-d42c679c03e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:25.023196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e4042-64dc-4b4f-898e-d42c679c03e7 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:25.032330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1057/2114] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:25.585220 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb142d4-6d82-46c1-bc89-f4cca083dd66 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:25.587701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7eb142d4-6d82-46c1-bc89-f4cca083dd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:25.701462 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb142d4-6d82-46c1-bc89-f4cca083dd66 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:25.708297 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb481cb3-3117-4670-a403-2c58f1739519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:25.709192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb481cb3-3117-4670-a403-2c58f1739519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:25.709622 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb481cb3-3117-4670-a403-2c58f1739519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:25.709904 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb481cb3-3117-4670-a403-2c58f1739519 admin admin] PATH_INFO: `/v3/users/a201d7ceaa244a59a357637d7f4c754a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:25.711335 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb481cb3-3117-4670-a403-2c58f1739519 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a201d7ceaa244a59a357637d7f4c754a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:25.715941 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb481cb3-3117-4670-a403-2c58f1739519 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:25.798533 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-cb481cb3-3117-4670-a403-2c58f1739519 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a201d7ceaa244a59a357637d7f4c754a'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:25.811826 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-cb481cb3-3117-4670-a403-2c58f1739519 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a201d7ceaa244a59a357637d7f4c754a'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:25.812884 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-cb481cb3-3117-4670-a403-2c58f1739519 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a201d7ceaa244a59a357637d7f4c754a'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:25.834435 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-cb481cb3-3117-4670-a403-2c58f1739519 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a201d7ceaa244a59a357637d7f4c754a'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:25.835199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-cb481cb3-3117-4670-a403-2c58f1739519 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a201d7ceaa244a59a357637d7f4c754a'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:25.862113 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07655bf9-e59d-4217-b843-644cee09d371 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:25.862113 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07655bf9-e59d-4217-b843-644cee09d371 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:25.863936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07655bf9-e59d-4217-b843-644cee09d371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:25.867696 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1058/2115] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:25 2022] DELETE /identity/v3/users/a201d7ceaa244a59a357637d7f4c754a => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:25.888174 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-07655bf9-e59d-4217-b843-644cee09d371 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:25.890380 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-07655bf9-e59d-4217-b843-644cee09d371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:25.981066 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1058/2116] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:25.993881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb481cb3-3117-4670-a403-2c58f1739519 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:25.995403 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-cb481cb3-3117-4670-a403-2c58f1739519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:26.084235 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb481cb3-3117-4670-a403-2c58f1739519 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:26.088513 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a6d4c-a97a-4849-ab1c-eccbb76814d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:26.089694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a6d4c-a97a-4849-ab1c-eccbb76814d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:26.089972 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a6d4c-a97a-4849-ab1c-eccbb76814d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:26.090240 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a6d4c-a97a-4849-ab1c-eccbb76814d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:26.100102 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a6d4c-a97a-4849-ab1c-eccbb76814d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:26.101517 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a6d4c-a97a-4849-ab1c-eccbb76814d3 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:26.111778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1059/2117] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:26.378881 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07655bf9-e59d-4217-b843-644cee09d371 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:26.380350 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-07655bf9-e59d-4217-b843-644cee09d371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:26.407397 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07655bf9-e59d-4217-b843-644cee09d371 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:26.422502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e78931-58dc-456e-bfe5-8a70357bfba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:26.423411 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e78931-58dc-456e-bfe5-8a70357bfba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:26.423756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e78931-58dc-456e-bfe5-8a70357bfba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:26.424144 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e78931-58dc-456e-bfe5-8a70357bfba0 admin admin] PATH_INFO: `/v3/projects/fcbd57382c9d49dfad81e175ae6cfbb5` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:26.425534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e78931-58dc-456e-bfe5-8a70357bfba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcbd57382c9d49dfad81e175ae6cfbb5)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:26.429155 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e78931-58dc-456e-bfe5-8a70357bfba0 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:26.533805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-44e78931-58dc-456e-bfe5-8a70357bfba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcbd57382c9d49dfad81e175ae6cfbb5', 'request_id': 'req-44e78931-58dc-456e-bfe5-8a70357bfba0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:26.546711 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-44e78931-58dc-456e-bfe5-8a70357bfba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcbd57382c9d49dfad81e175ae6cfbb5', 'request_id': 'req-44e78931-58dc-456e-bfe5-8a70357bfba0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:26.559421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-44e78931-58dc-456e-bfe5-8a70357bfba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcbd57382c9d49dfad81e175ae6cfbb5', 'request_id': 'req-44e78931-58dc-456e-bfe5-8a70357bfba0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:26.582604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-44e78931-58dc-456e-bfe5-8a70357bfba0 admin admin] The token cache is being invalidate because project fcbd57382c9d49dfad81e175ae6cfbb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:26.583267 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-44e78931-58dc-456e-bfe5-8a70357bfba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:26.584154 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-44e78931-58dc-456e-bfe5-8a70357bfba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:26.586000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1059/2118] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:26 2022] DELETE /identity/v3/projects/fcbd57382c9d49dfad81e175ae6cfbb5 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:26.591712 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85507c5a-5b13-4935-b1bf-1334ad3aba89 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:26.592286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85507c5a-5b13-4935-b1bf-1334ad3aba89 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:26.592388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85507c5a-5b13-4935-b1bf-1334ad3aba89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:26.619957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-85507c5a-5b13-4935-b1bf-1334ad3aba89 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:26.621672 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-85507c5a-5b13-4935-b1bf-1334ad3aba89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:26.652310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:26.704666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1060/2119] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:26.712461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0e2e8-b23d-4a38-bd6f-06dad3b9ca66 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:26.712699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0e2e8-b23d-4a38-bd6f-06dad3b9ca66 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:26.712974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0e2e8-b23d-4a38-bd6f-06dad3b9ca66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:26.733494 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-47d0e2e8-b23d-4a38-bd6f-06dad3b9ca66 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:26.734716 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-47d0e2e8-b23d-4a38-bd6f-06dad3b9ca66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:26.742630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1060/2120] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:26.748075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85507c5a-5b13-4935-b1bf-1334ad3aba89 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:26.760808 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85507c5a-5b13-4935-b1bf-1334ad3aba89 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:26.767154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7ffa9-a765-4230-bf22-1e1969fb499d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:26.767763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7ffa9-a765-4230-bf22-1e1969fb499d admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:26.768060 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7ffa9-a765-4230-bf22-1e1969fb499d admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:26.768324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7ffa9-a765-4230-bf22-1e1969fb499d admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:26.768974 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7ffa9-a765-4230-bf22-1e1969fb499d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:26.769280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7ffa9-a765-4230-bf22-1e1969fb499d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:26.770897 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7ffa9-a765-4230-bf22-1e1969fb499d admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:26.782702 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1061/2121] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:26.787418 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0e2e8-b23d-4a38-bd6f-06dad3b9ca66 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:26.797524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0e2e8-b23d-4a38-bd6f-06dad3b9ca66 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:26.802318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3720d-3bf2-4c3a-9bc8-c903c3133802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:26.802950 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3720d-3bf2-4c3a-9bc8-c903c3133802 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:26.803225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3720d-3bf2-4c3a-9bc8-c903c3133802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:26.803600 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3720d-3bf2-4c3a-9bc8-c903c3133802 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:26.804301 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b3720d-3bf2-4c3a-9bc8-c903c3133802 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:26.806898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b3720d-3bf2-4c3a-9bc8-c903c3133802 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:26.839210 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1061/2122] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:26 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:26.844036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7ffa9-a765-4230-bf22-1e1969fb499d admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:26.854931 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7ffa9-a765-4230-bf22-1e1969fb499d admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:26.918261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7da587-3130-45d1-99ba-6ef0844a53fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:26.919027 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7da587-3130-45d1-99ba-6ef0844a53fc admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:26.919321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7da587-3130-45d1-99ba-6ef0844a53fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:26.919538 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7da587-3130-45d1-99ba-6ef0844a53fc admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:26.920523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7da587-3130-45d1-99ba-6ef0844a53fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:26.923893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7da587-3130-45d1-99ba-6ef0844a53fc admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:26.947584 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1062/2123] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:26 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:26.952308 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3720d-3bf2-4c3a-9bc8-c903c3133802 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:26.963886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3720d-3bf2-4c3a-9bc8-c903c3133802 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:26.969744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728aba72-75b3-488b-8bd4-14be955d2f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:26.970339 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728aba72-75b3-488b-8bd4-14be955d2f8d admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:26.970618 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728aba72-75b3-488b-8bd4-14be955d2f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:26.970991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728aba72-75b3-488b-8bd4-14be955d2f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:26.971557 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728aba72-75b3-488b-8bd4-14be955d2f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:26.973197 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728aba72-75b3-488b-8bd4-14be955d2f8d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:26.984275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1062/2124] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:26.988522 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7da587-3130-45d1-99ba-6ef0844a53fc admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:26.998752 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7da587-3130-45d1-99ba-6ef0844a53fc admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:27.003439 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47357b3e-caed-4ddd-8990-5c267ed38b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:27.004103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47357b3e-caed-4ddd-8990-5c267ed38b13 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:27.004387 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47357b3e-caed-4ddd-8990-5c267ed38b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:27.004663 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47357b3e-caed-4ddd-8990-5c267ed38b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:27.005308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47357b3e-caed-4ddd-8990-5c267ed38b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:27.006940 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47357b3e-caed-4ddd-8990-5c267ed38b13 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:27.018142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1063/2125] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:27.022238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728aba72-75b3-488b-8bd4-14be955d2f8d admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:27.029490 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:27.034882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728aba72-75b3-488b-8bd4-14be955d2f8d admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:27.046612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c444f-15bc-4f0e-84a3-3c7fe8a710a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:27.047426 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c444f-15bc-4f0e-84a3-3c7fe8a710a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:27.047772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c444f-15bc-4f0e-84a3-3c7fe8a710a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:27.048135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c444f-15bc-4f0e-84a3-3c7fe8a710a2 admin admin] PATH_INFO: `/v3/projects/5d01387dc79c4b53b50119359bbeaab8/users/f625ddec3c3b406dbfdb8dfadd50db6e/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:27.064821 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c444f-15bc-4f0e-84a3-3c7fe8a710a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d01387dc79c4b53b50119359bbeaab8, user_id=f625ddec3c3b406dbfdb8dfadd50db6e, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:27.067519 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c444f-15bc-4f0e-84a3-3c7fe8a710a2 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:27.079979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1063/2126] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:27 2022] PUT /identity/v3/projects/5d01387dc79c4b53b50119359bbeaab8/users/f625ddec3c3b406dbfdb8dfadd50db6e/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:27.092396 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f6ae6-51b5-4f26-a5e5-b7cc6e50afe0 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:27.092763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f6ae6-51b5-4f26-a5e5-b7cc6e50afe0 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:27.093132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f6ae6-51b5-4f26-a5e5-b7cc6e50afe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:27.123985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-658f6ae6-51b5-4f26-a5e5-b7cc6e50afe0 None None] MFA Rules not processed for user `f625ddec3c3b406dbfdb8dfadd50db6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:27.125211 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-658f6ae6-51b5-4f26-a5e5-b7cc6e50afe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:27.203381 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1064/2127] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:27.207907 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c444f-15bc-4f0e-84a3-3c7fe8a710a2 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:27.271727 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c444f-15bc-4f0e-84a3-3c7fe8a710a2 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:27.277344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5dafa-97cc-4435-bd4b-d2eda3471e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:27.277959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5dafa-97cc-4435-bd4b-d2eda3471e3e admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:27.278234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5dafa-97cc-4435-bd4b-d2eda3471e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:27.278502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5dafa-97cc-4435-bd4b-d2eda3471e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:27.279203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5dafa-97cc-4435-bd4b-d2eda3471e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:27.281093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5dafa-97cc-4435-bd4b-d2eda3471e3e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:27.308552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1064/2128] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:27 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:27.312710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f6ae6-51b5-4f26-a5e5-b7cc6e50afe0 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:27.325046 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f6ae6-51b5-4f26-a5e5-b7cc6e50afe0 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:27.337987 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:27.382916 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24937f0a-29b5-43d6-befa-b934f7efb391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:27.383523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24937f0a-29b5-43d6-befa-b934f7efb391 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:27.384500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24937f0a-29b5-43d6-befa-b934f7efb391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:27.384805 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24937f0a-29b5-43d6-befa-b934f7efb391 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:27.385474 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24937f0a-29b5-43d6-befa-b934f7efb391 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:27.387412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24937f0a-29b5-43d6-befa-b934f7efb391 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:27.411841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1065/2129] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:27 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:27.415848 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5dafa-97cc-4435-bd4b-d2eda3471e3e admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:27.425912 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5dafa-97cc-4435-bd4b-d2eda3471e3e admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:27.430414 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e9e14-e8a6-43d4-b7d2-4e9d8e0cc622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:27.431085 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e9e14-e8a6-43d4-b7d2-4e9d8e0cc622 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:27.431357 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e9e14-e8a6-43d4-b7d2-4e9d8e0cc622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:27.431623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e9e14-e8a6-43d4-b7d2-4e9d8e0cc622 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:27.432258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e9e14-e8a6-43d4-b7d2-4e9d8e0cc622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:27.433829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e9e14-e8a6-43d4-b7d2-4e9d8e0cc622 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:27.444349 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1065/2130] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:27.448798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24937f0a-29b5-43d6-befa-b934f7efb391 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:27.459081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24937f0a-29b5-43d6-befa-b934f7efb391 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:27.464275 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836534e9-f85a-4b01-900e-5c2eda51e6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:27.465123 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836534e9-f85a-4b01-900e-5c2eda51e6a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:27.465406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836534e9-f85a-4b01-900e-5c2eda51e6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:27.465684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836534e9-f85a-4b01-900e-5c2eda51e6a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:27.466327 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836534e9-f85a-4b01-900e-5c2eda51e6a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:27.467956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836534e9-f85a-4b01-900e-5c2eda51e6a5 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:27.477960 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1066/2131] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:27.482114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e9e14-e8a6-43d4-b7d2-4e9d8e0cc622 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:27.492090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e9e14-e8a6-43d4-b7d2-4e9d8e0cc622 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:27.496899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76283fad-1682-479d-b9f9-b1d5e31175ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:27.497546 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76283fad-1682-479d-b9f9-b1d5e31175ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:27.497819 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76283fad-1682-479d-b9f9-b1d5e31175ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:27.498412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76283fad-1682-479d-b9f9-b1d5e31175ba admin admin] PATH_INFO: `/v3/projects/58800a8f5a73469d850d3653830a7847/users/b9942560ba704b869876c824d410fe52/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:27.514042 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76283fad-1682-479d-b9f9-b1d5e31175ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=58800a8f5a73469d850d3653830a7847, user_id=b9942560ba704b869876c824d410fe52, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:27.516702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76283fad-1682-479d-b9f9-b1d5e31175ba admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:27.530841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1066/2132] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:27 2022] PUT /identity/v3/projects/58800a8f5a73469d850d3653830a7847/users/b9942560ba704b869876c824d410fe52/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:27.541131 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816c9a9-7e7e-4de3-ae6e-996828fba8a1 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:27.541451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816c9a9-7e7e-4de3-ae6e-996828fba8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:27.541732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816c9a9-7e7e-4de3-ae6e-996828fba8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:27.574170 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-3816c9a9-7e7e-4de3-ae6e-996828fba8a1 None None] MFA Rules not processed for user `b9942560ba704b869876c824d410fe52`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:27.575489 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3816c9a9-7e7e-4de3-ae6e-996828fba8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:27.646801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1067/2133] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:27.651734 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76283fad-1682-479d-b9f9-b1d5e31175ba admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:27.714597 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816c9a9-7e7e-4de3-ae6e-996828fba8a1 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:27.716191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3816c9a9-7e7e-4de3-ae6e-996828fba8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:27.753234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76283fad-1682-479d-b9f9-b1d5e31175ba admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:27.761077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd4508-5ec2-4144-b335-1eca83fe963d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:27.761852 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd4508-5ec2-4144-b335-1eca83fe963d admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:27.762136 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd4508-5ec2-4144-b335-1eca83fe963d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:27.762410 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd4508-5ec2-4144-b335-1eca83fe963d admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:27.763874 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd4508-5ec2-4144-b335-1eca83fe963d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:27.766283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd4508-5ec2-4144-b335-1eca83fe963d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:27.805998 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1067/2134] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:27 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:27.808846 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816c9a9-7e7e-4de3-ae6e-996828fba8a1 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:27.810234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd4508-5ec2-4144-b335-1eca83fe963d admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:27.830910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:27.849642 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:27.889035 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85a1f2-9324-4415-9908-1906be663d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:27.889297 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd4508-5ec2-4144-b335-1eca83fe963d admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:27.889681 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85a1f2-9324-4415-9908-1906be663d5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:27.889964 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85a1f2-9324-4415-9908-1906be663d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:27.890236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85a1f2-9324-4415-9908-1906be663d5c admin admin] PATH_INFO: `/v3/users/63e15dd9ed344b74acfeb206bfe690f2` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:27.891387 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85a1f2-9324-4415-9908-1906be663d5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=63e15dd9ed344b74acfeb206bfe690f2)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:27.893465 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85a1f2-9324-4415-9908-1906be663d5c admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:27.894104 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4548d-88ad-4a1b-84fd-ffd244e064eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:27.894710 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4548d-88ad-4a1b-84fd-ffd244e064eb admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:27.895024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4548d-88ad-4a1b-84fd-ffd244e064eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:27.895297 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4548d-88ad-4a1b-84fd-ffd244e064eb admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:27.895926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4548d-88ad-4a1b-84fd-ffd244e064eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:27.897829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4548d-88ad-4a1b-84fd-ffd244e064eb admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:27.920520 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1068/2135] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:27 2022] POST /identity/v3/users => generated 298 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:27.924905 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4548d-88ad-4a1b-84fd-ffd244e064eb admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:27.935840 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4548d-88ad-4a1b-84fd-ffd244e064eb admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:27.940533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c43513-87e4-4128-b1a7-fb1c6f0d0925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:27.941127 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c43513-87e4-4128-b1a7-fb1c6f0d0925 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:27.941405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c43513-87e4-4128-b1a7-fb1c6f0d0925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:27.941679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c43513-87e4-4128-b1a7-fb1c6f0d0925 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:27.942326 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c43513-87e4-4128-b1a7-fb1c6f0d0925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:27.942896 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-db85a1f2-9324-4415-9908-1906be663d5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63e15dd9ed344b74acfeb206bfe690f2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:27.943943 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c43513-87e4-4128-b1a7-fb1c6f0d0925 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:27.951515 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-db85a1f2-9324-4415-9908-1906be663d5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63e15dd9ed344b74acfeb206bfe690f2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:27.952222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-db85a1f2-9324-4415-9908-1906be663d5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63e15dd9ed344b74acfeb206bfe690f2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:27.956538 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1069/2136] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:59:27.960609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c43513-87e4-4128-b1a7-fb1c6f0d0925 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:27.965755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-db85a1f2-9324-4415-9908-1906be663d5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63e15dd9ed344b74acfeb206bfe690f2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:27.966456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-db85a1f2-9324-4415-9908-1906be663d5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63e15dd9ed344b74acfeb206bfe690f2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:27.970882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c43513-87e4-4128-b1a7-fb1c6f0d0925 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:27.975593 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d88fe0-edc2-448d-b5cb-d3a957141a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:27.976239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d88fe0-edc2-448d-b5cb-d3a957141a5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:27.976517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d88fe0-edc2-448d-b5cb-d3a957141a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:27.976785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d88fe0-edc2-448d-b5cb-d3a957141a5f admin admin] PATH_INFO: `/v3/projects/cdf8964872b8474ba0c96e0117879fdb/users/0f7a5a5d2ef1478599095eda60e68670/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:27.978963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1068/2137] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:27 2022] DELETE /identity/v3/users/63e15dd9ed344b74acfeb206bfe690f2 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:27.984206 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af732053-9eee-487e-8b1f-ecc998a67592 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:27.985022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af732053-9eee-487e-8b1f-ecc998a67592 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:27.985022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af732053-9eee-487e-8b1f-ecc998a67592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:27.995578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d88fe0-edc2-448d-b5cb-d3a957141a5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf8964872b8474ba0c96e0117879fdb, user_id=0f7a5a5d2ef1478599095eda60e68670, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:27.998235 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d88fe0-edc2-448d-b5cb-d3a957141a5f admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:28.004375 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-af732053-9eee-487e-8b1f-ecc998a67592 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:28.005558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-af732053-9eee-487e-8b1f-ecc998a67592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:28.017780 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1070/2138] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:27 2022] PUT /identity/v3/projects/cdf8964872b8474ba0c96e0117879fdb/users/0f7a5a5d2ef1478599095eda60e68670/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:28.024959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d88fe0-edc2-448d-b5cb-d3a957141a5f admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:28.026773 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-85d88fe0-edc2-448d-b5cb-d3a957141a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:28.077826 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1069/2139] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:28.087958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af732053-9eee-487e-8b1f-ecc998a67592 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:28.089773 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-af732053-9eee-487e-8b1f-ecc998a67592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:28.103054 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d88fe0-edc2-448d-b5cb-d3a957141a5f admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:28.108024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9fee6-3817-4df0-ad52-e72250bd9120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:28.108719 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9fee6-3817-4df0-ad52-e72250bd9120 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:28.109366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9fee6-3817-4df0-ad52-e72250bd9120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:28.109574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9fee6-3817-4df0-ad52-e72250bd9120 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:28.110233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9fee6-3817-4df0-ad52-e72250bd9120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:28.111878 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9fee6-3817-4df0-ad52-e72250bd9120 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:28.130532 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1071/2140] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:28.134694 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9fee6-3817-4df0-ad52-e72250bd9120 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:28.148088 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9fee6-3817-4df0-ad52-e72250bd9120 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:28.153235 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b026984f-7bb7-446d-b71d-a3a5f871dd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:28.154042 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026984f-7bb7-446d-b71d-a3a5f871dd0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:28.154443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026984f-7bb7-446d-b71d-a3a5f871dd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:28.154846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026984f-7bb7-446d-b71d-a3a5f871dd0a admin admin] PATH_INFO: `/v3/projects/cdf8964872b8474ba0c96e0117879fdb/users/0f7a5a5d2ef1478599095eda60e68670/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:28.156940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b026984f-7bb7-446d-b71d-a3a5f871dd0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf8964872b8474ba0c96e0117879fdb, user_id=0f7a5a5d2ef1478599095eda60e68670, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:28.160016 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b026984f-7bb7-446d-b71d-a3a5f871dd0a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:28.172641 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1072/2141] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:28 2022] PUT /identity/v3/projects/cdf8964872b8474ba0c96e0117879fdb/users/0f7a5a5d2ef1478599095eda60e68670/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:28.184322 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eacce7-dfd8-4b53-a205-d1d86e395f56 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:28.184743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eacce7-dfd8-4b53-a205-d1d86e395f56 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:28.185118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eacce7-dfd8-4b53-a205-d1d86e395f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:28.190671 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af732053-9eee-487e-8b1f-ecc998a67592 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:28.198777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:28.244877 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-b4eacce7-dfd8-4b53-a205-d1d86e395f56 None None] MFA Rules not processed for user `0f7a5a5d2ef1478599095eda60e68670`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:28.244877 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b4eacce7-dfd8-4b53-a205-d1d86e395f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:28.274594 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa76dc-567c-4984-ab38-f2b6502fe5c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:28.275181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa76dc-567c-4984-ab38-f2b6502fe5c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:28.275644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa76dc-567c-4984-ab38-f2b6502fe5c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:28.276015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa76dc-567c-4984-ab38-f2b6502fe5c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:28.297312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:28.356739 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1073/2142] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:28.368723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eacce7-dfd8-4b53-a205-d1d86e395f56 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:28.370935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b4eacce7-dfd8-4b53-a205-d1d86e395f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:28.384738 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa76dc-567c-4984-ab38-f2b6502fe5c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:28.387771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa76dc-567c-4984-ab38-f2b6502fe5c1 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:28.398986 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1070/2143] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:28.452261 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eacce7-dfd8-4b53-a205-d1d86e395f56 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:28.457706 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1345acd-a814-440e-ba4f-cdf8bdab2c53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:28.459704 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1345acd-a814-440e-ba4f-cdf8bdab2c53 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:28.460431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1345acd-a814-440e-ba4f-cdf8bdab2c53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:28.460858 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1345acd-a814-440e-ba4f-cdf8bdab2c53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:28.463745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a1345acd-a814-440e-ba4f-cdf8bdab2c53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:28.537587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1345acd-a814-440e-ba4f-cdf8bdab2c53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:28.538972 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1345acd-a814-440e-ba4f-cdf8bdab2c53 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:28.550008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1074/2144] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:59:29.495441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa76dc-567c-4984-ab38-f2b6502fe5c1 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:29.497372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-adfa76dc-567c-4984-ab38-f2b6502fe5c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:29.526794 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa76dc-567c-4984-ab38-f2b6502fe5c1 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:29.526794 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e0bc8-abdd-47f7-ab7b-484538ece4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:29.526794 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e0bc8-abdd-47f7-ab7b-484538ece4fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:29.526794 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e0bc8-abdd-47f7-ab7b-484538ece4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:29.528110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e0bc8-abdd-47f7-ab7b-484538ece4fe admin admin] PATH_INFO: `/v3/projects/1747e74c003b4fbabef2d6d827e2a555` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:29.528110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e0bc8-abdd-47f7-ab7b-484538ece4fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=1747e74c003b4fbabef2d6d827e2a555)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:29.530762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e0bc8-abdd-47f7-ab7b-484538ece4fe admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:29.644482 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-382e0bc8-abdd-47f7-ab7b-484538ece4fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1747e74c003b4fbabef2d6d827e2a555', 'request_id': 'req-382e0bc8-abdd-47f7-ab7b-484538ece4fe'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:29.660165 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-382e0bc8-abdd-47f7-ab7b-484538ece4fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1747e74c003b4fbabef2d6d827e2a555', 'request_id': 'req-382e0bc8-abdd-47f7-ab7b-484538ece4fe'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:29.675583 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-382e0bc8-abdd-47f7-ab7b-484538ece4fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1747e74c003b4fbabef2d6d827e2a555', 'request_id': 'req-382e0bc8-abdd-47f7-ab7b-484538ece4fe'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:29.700347 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-382e0bc8-abdd-47f7-ab7b-484538ece4fe admin admin] The token cache is being invalidate because project 1747e74c003b4fbabef2d6d827e2a555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:29.700723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-382e0bc8-abdd-47f7-ab7b-484538ece4fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:29.701712 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-382e0bc8-abdd-47f7-ab7b-484538ece4fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:29.703285 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1071/2145] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:29 2022] DELETE /identity/v3/projects/1747e74c003b4fbabef2d6d827e2a555 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:29.708282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5af17-1054-4414-bcfe-74a5e269332b None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:29.708813 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5af17-1054-4414-bcfe-74a5e269332b None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:29.708906 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5af17-1054-4414-bcfe-74a5e269332b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:29.730567 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-47b5af17-1054-4414-bcfe-74a5e269332b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:29.731912 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-47b5af17-1054-4414-bcfe-74a5e269332b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:29.794330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1075/2146] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:29.801971 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351eb3a4-9652-405c-9efa-2d2947229a7b None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:29.802253 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351eb3a4-9652-405c-9efa-2d2947229a7b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:29.802529 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351eb3a4-9652-405c-9efa-2d2947229a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:29.822407 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-351eb3a4-9652-405c-9efa-2d2947229a7b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:29.823611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-351eb3a4-9652-405c-9efa-2d2947229a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:29.832293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1072/2147] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:29.836820 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5af17-1054-4414-bcfe-74a5e269332b None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:29.847745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5af17-1054-4414-bcfe-74a5e269332b None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:29.853059 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b65ec-8251-44f0-8568-52e2608d8715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:29.853678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b65ec-8251-44f0-8568-52e2608d8715 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:29.854008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b65ec-8251-44f0-8568-52e2608d8715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:29.854229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b65ec-8251-44f0-8568-52e2608d8715 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:29.854919 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b65ec-8251-44f0-8568-52e2608d8715 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:29.855226 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b65ec-8251-44f0-8568-52e2608d8715 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:29.856832 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b65ec-8251-44f0-8568-52e2608d8715 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:29.869807 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1076/2148] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:59:29.873975 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351eb3a4-9652-405c-9efa-2d2947229a7b None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:29.883242 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351eb3a4-9652-405c-9efa-2d2947229a7b None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:29.887958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e288cf-0cec-40e6-857e-9fa667969b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:29.888555 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e288cf-0cec-40e6-857e-9fa667969b27 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:29.888842 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e288cf-0cec-40e6-857e-9fa667969b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:29.889114 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e288cf-0cec-40e6-857e-9fa667969b27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:29.889746 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e288cf-0cec-40e6-857e-9fa667969b27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:29.891676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e288cf-0cec-40e6-857e-9fa667969b27 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:29.924652 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1073/2149] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:29 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:29.928584 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b65ec-8251-44f0-8568-52e2608d8715 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:29.939388 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b65ec-8251-44f0-8568-52e2608d8715 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:29.959421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:30.007877 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3830b7c-2d4b-45bb-bf17-f97734adfe02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:30.009008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3830b7c-2d4b-45bb-bf17-f97734adfe02 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:30.009498 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3830b7c-2d4b-45bb-bf17-f97734adfe02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:30.009989 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3830b7c-2d4b-45bb-bf17-f97734adfe02 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:30.010921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3830b7c-2d4b-45bb-bf17-f97734adfe02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:30.013047 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3830b7c-2d4b-45bb-bf17-f97734adfe02 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:30.036615 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1077/2150] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:29 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:30.048543 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e288cf-0cec-40e6-857e-9fa667969b27 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:30.063174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e288cf-0cec-40e6-857e-9fa667969b27 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:30.069234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003d193-27d0-4205-9fa1-c77d22f280fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:30.069876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003d193-27d0-4205-9fa1-c77d22f280fd admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:30.070160 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003d193-27d0-4205-9fa1-c77d22f280fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:30.070535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003d193-27d0-4205-9fa1-c77d22f280fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:30.071319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003d193-27d0-4205-9fa1-c77d22f280fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:30.072991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003d193-27d0-4205-9fa1-c77d22f280fd admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:30.079012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:30.090341 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3830b7c-2d4b-45bb-bf17-f97734adfe02 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:30.095852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1074/2151] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:30.101903 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3830b7c-2d4b-45bb-bf17-f97734adfe02 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:30.108084 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562a777-a6a0-4497-8d63-2927bbf47ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:30.108498 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562a777-a6a0-4497-8d63-2927bbf47ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:30.108678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562a777-a6a0-4497-8d63-2927bbf47ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:30.108978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562a777-a6a0-4497-8d63-2927bbf47ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:30.109527 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562a777-a6a0-4497-8d63-2927bbf47ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:30.111149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562a777-a6a0-4497-8d63-2927bbf47ac5 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:30.127188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003d193-27d0-4205-9fa1-c77d22f280fd admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:30.127688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1078/2152] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 14:59:30.137782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003d193-27d0-4205-9fa1-c77d22f280fd admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:30.147438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf06350-c5d2-4017-9497-9b8ff06ff8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:30.147438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf06350-c5d2-4017-9497-9b8ff06ff8d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:30.147438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf06350-c5d2-4017-9497-9b8ff06ff8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:30.147438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf06350-c5d2-4017-9497-9b8ff06ff8d7 admin admin] PATH_INFO: `/v3/projects/cd727632a3ba47c28b4a296b4a2eaae6/users/44ee437ebad14039b43e3157410b8654/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:30.163827 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf06350-c5d2-4017-9497-9b8ff06ff8d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd727632a3ba47c28b4a296b4a2eaae6, user_id=44ee437ebad14039b43e3157410b8654, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:30.167456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf06350-c5d2-4017-9497-9b8ff06ff8d7 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:30.179678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562a777-a6a0-4497-8d63-2927bbf47ac5 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:30.181613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-6562a777-a6a0-4497-8d63-2927bbf47ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:30.183644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1075/2153] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:30 2022] PUT /identity/v3/projects/cd727632a3ba47c28b4a296b4a2eaae6/users/44ee437ebad14039b43e3157410b8654/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:30.196088 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae8934-8c4c-4bcd-990b-8ef108c1e4ce None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:30.196412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae8934-8c4c-4bcd-990b-8ef108c1e4ce None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:30.196736 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae8934-8c4c-4bcd-990b-8ef108c1e4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:30.239182 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-2bae8934-8c4c-4bcd-990b-8ef108c1e4ce None None] MFA Rules not processed for user `44ee437ebad14039b43e3157410b8654`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:30.240158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-2bae8934-8c4c-4bcd-990b-8ef108c1e4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:30.264314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562a777-a6a0-4497-8d63-2927bbf47ac5 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:30.319701 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce9cc28-cc6f-448a-b14a-24afa4427301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:30.320340 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9cc28-cc6f-448a-b14a-24afa4427301 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:30.320612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9cc28-cc6f-448a-b14a-24afa4427301 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:30.321753 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce9cc28-cc6f-448a-b14a-24afa4427301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:30.323492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-8ce9cc28-cc6f-448a-b14a-24afa4427301 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:30.325866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1076/2154] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:30.337551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae8934-8c4c-4bcd-990b-8ef108c1e4ce None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:30.339103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-2bae8934-8c4c-4bcd-990b-8ef108c1e4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:30.352545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae8934-8c4c-4bcd-990b-8ef108c1e4ce None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:30.356267 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3efef-f0c5-4721-9cf5-e13c8ba672b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:30.356988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3efef-f0c5-4721-9cf5-e13c8ba672b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:30.357340 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3efef-f0c5-4721-9cf5-e13c8ba672b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:30.357560 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3efef-f0c5-4721-9cf5-e13c8ba672b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:30.367550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3efef-f0c5-4721-9cf5-e13c8ba672b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:30.369039 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3efef-f0c5-4721-9cf5-e13c8ba672b5 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:30.378000 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1077/2155] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:30.393684 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce9cc28-cc6f-448a-b14a-24afa4427301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:30.395150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce9cc28-cc6f-448a-b14a-24afa4427301 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:30.403067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3efef-f0c5-4721-9cf5-e13c8ba672b5 service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:30.407944 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1079/2156] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:59:30.467009 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3efef-f0c5-4721-9cf5-e13c8ba672b5 service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:30.471669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70039a0-710a-49cc-912d-207983faed8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:30.472332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70039a0-710a-49cc-912d-207983faed8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:30.472603 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70039a0-710a-49cc-912d-207983faed8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:30.472874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70039a0-710a-49cc-912d-207983faed8f admin admin] PATH_INFO: `/v3/users/44ee437ebad14039b43e3157410b8654` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:30.474016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70039a0-710a-49cc-912d-207983faed8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=44ee437ebad14039b43e3157410b8654)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:30.475962 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70039a0-710a-49cc-912d-207983faed8f admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:30.533804 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e70039a0-710a-49cc-912d-207983faed8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44ee437ebad14039b43e3157410b8654'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:30.543285 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e70039a0-710a-49cc-912d-207983faed8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44ee437ebad14039b43e3157410b8654'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:30.544071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e70039a0-710a-49cc-912d-207983faed8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44ee437ebad14039b43e3157410b8654'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:30.545738 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:30.566006 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e70039a0-710a-49cc-912d-207983faed8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44ee437ebad14039b43e3157410b8654'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:30.567137 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e70039a0-710a-49cc-912d-207983faed8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44ee437ebad14039b43e3157410b8654'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:30.583688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1078/2157] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:30 2022] DELETE /identity/v3/users/44ee437ebad14039b43e3157410b8654 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:30.588360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca35a7a-d20e-4774-990f-3f615f5248ab None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:30.588721 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca35a7a-d20e-4774-990f-3f615f5248ab None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:30.589935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca35a7a-d20e-4774-990f-3f615f5248ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:30.611293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-fca35a7a-d20e-4774-990f-3f615f5248ab None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:30.612518 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-fca35a7a-d20e-4774-990f-3f615f5248ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:30.616696 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:30.706126 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1080/2158] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:30.719861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70039a0-710a-49cc-912d-207983faed8f admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:30.721725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e70039a0-710a-49cc-912d-207983faed8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:30.799952 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70039a0-710a-49cc-912d-207983faed8f admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:30.804208 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3d08c-8254-41d1-a6b2-73b72abd15ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:30.804860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3d08c-8254-41d1-a6b2-73b72abd15ea service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:30.805138 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3d08c-8254-41d1-a6b2-73b72abd15ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:30.805709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3d08c-8254-41d1-a6b2-73b72abd15ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:30.816446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3d08c-8254-41d1-a6b2-73b72abd15ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:30.818068 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3d08c-8254-41d1-a6b2-73b72abd15ea service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:30.829549 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1079/2159] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:31.123283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca35a7a-d20e-4774-990f-3f615f5248ab None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:31.126347 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-fca35a7a-d20e-4774-990f-3f615f5248ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:31.192505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3d08c-8254-41d1-a6b2-73b72abd15ea service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:31.194478 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-cfa3d08c-8254-41d1-a6b2-73b72abd15ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:31.194613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca35a7a-d20e-4774-990f-3f615f5248ab None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:31.199997 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03f1d7-5083-4538-8f51-7e1ba183f860 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:31.201301 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03f1d7-5083-4538-8f51-7e1ba183f860 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:31.201712 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03f1d7-5083-4538-8f51-7e1ba183f860 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:31.202134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03f1d7-5083-4538-8f51-7e1ba183f860 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:31.204078 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-8e03f1d7-5083-4538-8f51-7e1ba183f860 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:31.213550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3d08c-8254-41d1-a6b2-73b72abd15ea service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:31.232109 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77e30d-2860-40c2-8421-2e7dc1d82c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:31.232784 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77e30d-2860-40c2-8421-2e7dc1d82c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:31.233066 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77e30d-2860-40c2-8421-2e7dc1d82c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:31.233321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77e30d-2860-40c2-8421-2e7dc1d82c92 admin admin] PATH_INFO: `/v3/projects/cd727632a3ba47c28b4a296b4a2eaae6` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:31.234615 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77e30d-2860-40c2-8421-2e7dc1d82c92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd727632a3ba47c28b4a296b4a2eaae6)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:31.236541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77e30d-2860-40c2-8421-2e7dc1d82c92 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:31.289589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03f1d7-5083-4538-8f51-7e1ba183f860 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:31.291118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03f1d7-5083-4538-8f51-7e1ba183f860 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:31.308964 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5a77e30d-2860-40c2-8421-2e7dc1d82c92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd727632a3ba47c28b4a296b4a2eaae6', 'request_id': 'req-5a77e30d-2860-40c2-8421-2e7dc1d82c92'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:31.309954 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1081/2160] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:31.324265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5a77e30d-2860-40c2-8421-2e7dc1d82c92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd727632a3ba47c28b4a296b4a2eaae6', 'request_id': 'req-5a77e30d-2860-40c2-8421-2e7dc1d82c92'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:31.334549 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5a77e30d-2860-40c2-8421-2e7dc1d82c92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd727632a3ba47c28b4a296b4a2eaae6', 'request_id': 'req-5a77e30d-2860-40c2-8421-2e7dc1d82c92'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:31.347697 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5a77e30d-2860-40c2-8421-2e7dc1d82c92 admin admin] The token cache is being invalidate because project cd727632a3ba47c28b4a296b4a2eaae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:31.347988 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5a77e30d-2860-40c2-8421-2e7dc1d82c92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:31.348989 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5a77e30d-2860-40c2-8421-2e7dc1d82c92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:31.351118 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1080/2161] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:31 2022] DELETE /identity/v3/projects/cd727632a3ba47c28b4a296b4a2eaae6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:31.357163 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62213fb8-6845-4780-8b37-1d5b0000358b None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:31.357837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62213fb8-6845-4780-8b37-1d5b0000358b None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:31.358405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62213fb8-6845-4780-8b37-1d5b0000358b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:31.380083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-62213fb8-6845-4780-8b37-1d5b0000358b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:31.381300 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-62213fb8-6845-4780-8b37-1d5b0000358b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:31.441203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:31.450831 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1082/2162] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:31.458544 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76b0aa-170a-4287-850f-db80569d2bc3 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:31.458849 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76b0aa-170a-4287-850f-db80569d2bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:31.459125 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76b0aa-170a-4287-850f-db80569d2bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:31.480543 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-3b76b0aa-170a-4287-850f-db80569d2bc3 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:31.481747 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3b76b0aa-170a-4287-850f-db80569d2bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:31.489612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1081/2163] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:31.495032 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62213fb8-6845-4780-8b37-1d5b0000358b None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:31.504938 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62213fb8-6845-4780-8b37-1d5b0000358b None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:31.510120 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5e30f-4867-4d3e-bc07-21a0b133c3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:31.510716 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5e30f-4867-4d3e-bc07-21a0b133c3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:31.511021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5e30f-4867-4d3e-bc07-21a0b133c3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:31.511292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5e30f-4867-4d3e-bc07-21a0b133c3f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:31.511936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5e30f-4867-4d3e-bc07-21a0b133c3f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:31.512235 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5e30f-4867-4d3e-bc07-21a0b133c3f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:31.513834 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5e30f-4867-4d3e-bc07-21a0b133c3f8 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:31.525590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1083/2164] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 14:59:31.529874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76b0aa-170a-4287-850f-db80569d2bc3 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:31.539137 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76b0aa-170a-4287-850f-db80569d2bc3 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:31.543993 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e823f-ac03-4345-87a0-9911d3f11319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:31.544588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e823f-ac03-4345-87a0-9911d3f11319 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:31.544864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e823f-ac03-4345-87a0-9911d3f11319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:31.545140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e823f-ac03-4345-87a0-9911d3f11319 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:31.545871 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e823f-ac03-4345-87a0-9911d3f11319 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:31.547850 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e823f-ac03-4345-87a0-9911d3f11319 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:31.578308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1082/2165] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:31 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:31.582931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5e30f-4867-4d3e-bc07-21a0b133c3f8 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:31.593492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5e30f-4867-4d3e-bc07-21a0b133c3f8 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:31.663367 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f8239-52a4-4567-ae2d-4247779994f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:31.663985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f8239-52a4-4567-ae2d-4247779994f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:31.664261 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f8239-52a4-4567-ae2d-4247779994f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:31.664531 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f8239-52a4-4567-ae2d-4247779994f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:31.665191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f8239-52a4-4567-ae2d-4247779994f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:31.667243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f8239-52a4-4567-ae2d-4247779994f1 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:31.691498 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1084/2166] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:31 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:31.695790 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e823f-ac03-4345-87a0-9911d3f11319 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:31.706676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e823f-ac03-4345-87a0-9911d3f11319 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:31.711418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aede98-08b3-478b-b9a2-6715490e83e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:31.712018 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aede98-08b3-478b-b9a2-6715490e83e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:31.712298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aede98-08b3-478b-b9a2-6715490e83e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:31.712568 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aede98-08b3-478b-b9a2-6715490e83e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:31.713206 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aede98-08b3-478b-b9a2-6715490e83e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:31.715155 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aede98-08b3-478b-b9a2-6715490e83e1 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:31.726763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1083/2167] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:31.729313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f8239-52a4-4567-ae2d-4247779994f1 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:31.740035 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f8239-52a4-4567-ae2d-4247779994f1 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:31.746641 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad89af-1e11-4b57-bf4e-5c5b1163da60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:31.747311 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad89af-1e11-4b57-bf4e-5c5b1163da60 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:31.747617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad89af-1e11-4b57-bf4e-5c5b1163da60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:31.747894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad89af-1e11-4b57-bf4e-5c5b1163da60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:31.748608 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad89af-1e11-4b57-bf4e-5c5b1163da60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:31.750252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad89af-1e11-4b57-bf4e-5c5b1163da60 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:31.763010 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1085/2168] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:31.767325 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aede98-08b3-478b-b9a2-6715490e83e1 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:31.780012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aede98-08b3-478b-b9a2-6715490e83e1 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:31.790414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f93893-44e3-4a7c-9f66-a58f0619debc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:31.791265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f93893-44e3-4a7c-9f66-a58f0619debc admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:31.791695 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f93893-44e3-4a7c-9f66-a58f0619debc admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:31.792235 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f93893-44e3-4a7c-9f66-a58f0619debc admin admin] PATH_INFO: `/v3/projects/412a2bb8e4e24a4db88f4ce940bf6a34/users/5b75e9af270641f8a96ed7a7be1e0c53/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:31.813474 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f93893-44e3-4a7c-9f66-a58f0619debc admin admin] RBAC: Authorizing `identity:create_grant(project_id=412a2bb8e4e24a4db88f4ce940bf6a34, user_id=5b75e9af270641f8a96ed7a7be1e0c53, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:31.816594 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f93893-44e3-4a7c-9f66-a58f0619debc admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:31.820444 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:31.837547 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1084/2169] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:31 2022] PUT /identity/v3/projects/412a2bb8e4e24a4db88f4ce940bf6a34/users/5b75e9af270641f8a96ed7a7be1e0c53/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:31.847996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c1f37-88c7-4ec0-8855-0f9342998f2a None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:31.848369 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c1f37-88c7-4ec0-8855-0f9342998f2a None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:31.848717 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c1f37-88c7-4ec0-8855-0f9342998f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:31.883898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-160c1f37-88c7-4ec0-8855-0f9342998f2a None None] MFA Rules not processed for user `5b75e9af270641f8a96ed7a7be1e0c53`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:31.885208 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-160c1f37-88c7-4ec0-8855-0f9342998f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:31.967404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1086/2170] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:31.995823 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f93893-44e3-4a7c-9f66-a58f0619debc admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:31.997794 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-21f93893-44e3-4a7c-9f66-a58f0619debc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:32.064393 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f93893-44e3-4a7c-9f66-a58f0619debc admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:32.068390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276ce27-61a9-431d-8445-4a7b5d7975a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:32.069005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276ce27-61a9-431d-8445-4a7b5d7975a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:32.069294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276ce27-61a9-431d-8445-4a7b5d7975a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:32.069612 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276ce27-61a9-431d-8445-4a7b5d7975a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:32.078585 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276ce27-61a9-431d-8445-4a7b5d7975a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:32.083493 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276ce27-61a9-431d-8445-4a7b5d7975a5 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:32.092704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1085/2171] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:33.470459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd0f04-92d3-45d2-b8e9-b6ca4c015424 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:33.471006 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd0f04-92d3-45d2-b8e9-b6ca4c015424 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:33.471114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd0f04-92d3-45d2-b8e9-b6ca4c015424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:33.493915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-85fd0f04-92d3-45d2-b8e9-b6ca4c015424 None None] MFA Rules not processed for user `5b75e9af270641f8a96ed7a7be1e0c53`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:33.495416 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-85fd0f04-92d3-45d2-b8e9-b6ca4c015424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:33.506551 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1087/2172] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:35.315554 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276ce27-61a9-431d-8445-4a7b5d7975a5 service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:35.317010 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-8276ce27-61a9-431d-8445-4a7b5d7975a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:35.414056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276ce27-61a9-431d-8445-4a7b5d7975a5 service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:35.420046 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c065dd-775a-4e43-b8a9-77f7282705f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:35.420801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c065dd-775a-4e43-b8a9-77f7282705f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:35.421140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c065dd-775a-4e43-b8a9-77f7282705f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:35.421503 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c065dd-775a-4e43-b8a9-77f7282705f6 admin admin] PATH_INFO: `/v3/users/f625ddec3c3b406dbfdb8dfadd50db6e` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:35.422765 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c065dd-775a-4e43-b8a9-77f7282705f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f625ddec3c3b406dbfdb8dfadd50db6e)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:35.424798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c065dd-775a-4e43-b8a9-77f7282705f6 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:35.486785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-97c065dd-775a-4e43-b8a9-77f7282705f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f625ddec3c3b406dbfdb8dfadd50db6e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.496261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-97c065dd-775a-4e43-b8a9-77f7282705f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f625ddec3c3b406dbfdb8dfadd50db6e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.496904 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-97c065dd-775a-4e43-b8a9-77f7282705f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f625ddec3c3b406dbfdb8dfadd50db6e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.512523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-97c065dd-775a-4e43-b8a9-77f7282705f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f625ddec3c3b406dbfdb8dfadd50db6e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.513414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-97c065dd-775a-4e43-b8a9-77f7282705f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f625ddec3c3b406dbfdb8dfadd50db6e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:35.531915 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1086/2173] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:35 2022] DELETE /identity/v3/users/f625ddec3c3b406dbfdb8dfadd50db6e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:35.536416 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71b8e2-682d-4343-9e45-519dd4e73998 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:35.537062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71b8e2-682d-4343-9e45-519dd4e73998 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:35.537062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71b8e2-682d-4343-9e45-519dd4e73998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:35.558913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-5b71b8e2-682d-4343-9e45-519dd4e73998 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:35.560635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-5b71b8e2-682d-4343-9e45-519dd4e73998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:35.619969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:35.649806 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1088/2174] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:35.658938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c065dd-775a-4e43-b8a9-77f7282705f6 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:35.662117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-97c065dd-775a-4e43-b8a9-77f7282705f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:35.686141 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:35.757873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c065dd-775a-4e43-b8a9-77f7282705f6 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:35.763543 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58d413-354b-4a07-ac20-e159cf1aa59b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:35.764483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58d413-354b-4a07-ac20-e159cf1aa59b service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:35.764483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58d413-354b-4a07-ac20-e159cf1aa59b service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:35.764769 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58d413-354b-4a07-ac20-e159cf1aa59b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:35.776711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58d413-354b-4a07-ac20-e159cf1aa59b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:35.778358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58d413-354b-4a07-ac20-e159cf1aa59b service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:35.790801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1087/2175] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:36.684447 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71b8e2-682d-4343-9e45-519dd4e73998 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:36.685888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-5b71b8e2-682d-4343-9e45-519dd4e73998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:36.720191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71b8e2-682d-4343-9e45-519dd4e73998 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:36.723175 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839bfb3-1163-4127-81ee-503bc534c521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:36.724191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839bfb3-1163-4127-81ee-503bc534c521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:36.724720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839bfb3-1163-4127-81ee-503bc534c521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:36.725193 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839bfb3-1163-4127-81ee-503bc534c521 admin admin] PATH_INFO: `/v3/projects/5d01387dc79c4b53b50119359bbeaab8` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:36.729244 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839bfb3-1163-4127-81ee-503bc534c521 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d01387dc79c4b53b50119359bbeaab8)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:36.730809 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839bfb3-1163-4127-81ee-503bc534c521 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:36.816533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1839bfb3-1163-4127-81ee-503bc534c521 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d01387dc79c4b53b50119359bbeaab8', 'request_id': 'req-1839bfb3-1163-4127-81ee-503bc534c521'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.823512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1839bfb3-1163-4127-81ee-503bc534c521 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d01387dc79c4b53b50119359bbeaab8', 'request_id': 'req-1839bfb3-1163-4127-81ee-503bc534c521'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.829875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1839bfb3-1163-4127-81ee-503bc534c521 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d01387dc79c4b53b50119359bbeaab8', 'request_id': 'req-1839bfb3-1163-4127-81ee-503bc534c521'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.851160 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1839bfb3-1163-4127-81ee-503bc534c521 admin admin] The token cache is being invalidate because project 5d01387dc79c4b53b50119359bbeaab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:36.851520 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1839bfb3-1163-4127-81ee-503bc534c521 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.852998 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1839bfb3-1163-4127-81ee-503bc534c521 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:36.855464 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1089/2176] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:36 2022] DELETE /identity/v3/projects/5d01387dc79c4b53b50119359bbeaab8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:36.859146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58d413-354b-4a07-ac20-e159cf1aa59b service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:36.870443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58d413-354b-4a07-ac20-e159cf1aa59b service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:36.944274 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c0b15-fb62-4f26-bff9-f8f289bd418b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:36.945276 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c0b15-fb62-4f26-bff9-f8f289bd418b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:36.945889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c0b15-fb62-4f26-bff9-f8f289bd418b admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:36.947201 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c0b15-fb62-4f26-bff9-f8f289bd418b admin admin] PATH_INFO: `/v3/users/b9942560ba704b869876c824d410fe52` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:36.948503 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c0b15-fb62-4f26-bff9-f8f289bd418b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9942560ba704b869876c824d410fe52)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:36.950861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6c0b15-fb62-4f26-bff9-f8f289bd418b admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.035508 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-fb6c0b15-fb62-4f26-bff9-f8f289bd418b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9942560ba704b869876c824d410fe52'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.049741 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-fb6c0b15-fb62-4f26-bff9-f8f289bd418b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9942560ba704b869876c824d410fe52'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.051030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-fb6c0b15-fb62-4f26-bff9-f8f289bd418b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9942560ba704b869876c824d410fe52'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.069868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-fb6c0b15-fb62-4f26-bff9-f8f289bd418b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9942560ba704b869876c824d410fe52'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.073033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-fb6c0b15-fb62-4f26-bff9-f8f289bd418b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9942560ba704b869876c824d410fe52'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.090085 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1088/2177] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:36 2022] DELETE /identity/v3/users/b9942560ba704b869876c824d410fe52 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:37.482011 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839bfb3-1163-4127-81ee-503bc534c521 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.483484 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1839bfb3-1163-4127-81ee-503bc534c521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:37.507752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839bfb3-1163-4127-81ee-503bc534c521 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.574606 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773b659-ce82-4fac-9aa1-22b2c73c6a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.575669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773b659-ce82-4fac-9aa1-22b2c73c6a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.575949 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773b659-ce82-4fac-9aa1-22b2c73c6a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.576219 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773b659-ce82-4fac-9aa1-22b2c73c6a21 admin admin] PATH_INFO: `/v3/projects/58800a8f5a73469d850d3653830a7847` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.577365 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773b659-ce82-4fac-9aa1-22b2c73c6a21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58800a8f5a73469d850d3653830a7847)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.579321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773b659-ce82-4fac-9aa1-22b2c73c6a21 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.631113 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-e773b659-ce82-4fac-9aa1-22b2c73c6a21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58800a8f5a73469d850d3653830a7847', 'request_id': 'req-e773b659-ce82-4fac-9aa1-22b2c73c6a21'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.637736 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-e773b659-ce82-4fac-9aa1-22b2c73c6a21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58800a8f5a73469d850d3653830a7847', 'request_id': 'req-e773b659-ce82-4fac-9aa1-22b2c73c6a21'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.643451 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-e773b659-ce82-4fac-9aa1-22b2c73c6a21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58800a8f5a73469d850d3653830a7847', 'request_id': 'req-e773b659-ce82-4fac-9aa1-22b2c73c6a21'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.657481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-e773b659-ce82-4fac-9aa1-22b2c73c6a21 admin admin] The token cache is being invalidate because project 58800a8f5a73469d850d3653830a7847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:37.657770 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-e773b659-ce82-4fac-9aa1-22b2c73c6a21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.658414 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-e773b659-ce82-4fac-9aa1-22b2c73c6a21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.660126 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1090/2178] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:37 2022] DELETE /identity/v3/projects/58800a8f5a73469d850d3653830a7847 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:37.665079 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c0b15-fb62-4f26-bff9-f8f289bd418b admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.668840 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-fb6c0b15-fb62-4f26-bff9-f8f289bd418b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:37.683632 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c0b15-fb62-4f26-bff9-f8f289bd418b admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:37.721598 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:37.750449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d1ad5-0586-425f-ac3f-db9a8c6d5385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:37.751131 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d1ad5-0586-425f-ac3f-db9a8c6d5385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:37.751414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d1ad5-0586-425f-ac3f-db9a8c6d5385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:37.751689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d1ad5-0586-425f-ac3f-db9a8c6d5385 admin admin] PATH_INFO: `/v3/users/0f7a5a5d2ef1478599095eda60e68670` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:37.752830 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d1ad5-0586-425f-ac3f-db9a8c6d5385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f7a5a5d2ef1478599095eda60e68670)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:37.756277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d1ad5-0586-425f-ac3f-db9a8c6d5385 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:37.807155 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2f9d1ad5-0586-425f-ac3f-db9a8c6d5385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f7a5a5d2ef1478599095eda60e68670'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.816643 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2f9d1ad5-0586-425f-ac3f-db9a8c6d5385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f7a5a5d2ef1478599095eda60e68670'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.817375 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2f9d1ad5-0586-425f-ac3f-db9a8c6d5385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f7a5a5d2ef1478599095eda60e68670'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.837023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2f9d1ad5-0586-425f-ac3f-db9a8c6d5385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f7a5a5d2ef1478599095eda60e68670'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.837771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2f9d1ad5-0586-425f-ac3f-db9a8c6d5385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f7a5a5d2ef1478599095eda60e68670'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:37.860402 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1089/2179] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:37 2022] DELETE /identity/v3/users/0f7a5a5d2ef1478599095eda60e68670 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:37.884591 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773b659-ce82-4fac-9aa1-22b2c73c6a21 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:37.886083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e773b659-ce82-4fac-9aa1-22b2c73c6a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:37.895477 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:37.909924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773b659-ce82-4fac-9aa1-22b2c73c6a21 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.014991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979981a3-be10-49d7-927f-f9cdf8a328bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.015890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979981a3-be10-49d7-927f-f9cdf8a328bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.019709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979981a3-be10-49d7-927f-f9cdf8a328bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.019709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979981a3-be10-49d7-927f-f9cdf8a328bb admin admin] PATH_INFO: `/v3/users/79184aa0a79a4d85b3674541b43332d0` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.019709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979981a3-be10-49d7-927f-f9cdf8a328bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=79184aa0a79a4d85b3674541b43332d0)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.034079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979981a3-be10-49d7-927f-f9cdf8a328bb admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.163161 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-979981a3-be10-49d7-927f-f9cdf8a328bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79184aa0a79a4d85b3674541b43332d0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.178282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-979981a3-be10-49d7-927f-f9cdf8a328bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79184aa0a79a4d85b3674541b43332d0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.204287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-979981a3-be10-49d7-927f-f9cdf8a328bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79184aa0a79a4d85b3674541b43332d0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.226946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-979981a3-be10-49d7-927f-f9cdf8a328bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79184aa0a79a4d85b3674541b43332d0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.228092 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-979981a3-be10-49d7-927f-f9cdf8a328bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79184aa0a79a4d85b3674541b43332d0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.245488 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1091/2180] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:37 2022] DELETE /identity/v3/users/79184aa0a79a4d85b3674541b43332d0 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:38.251836 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841cd99-55ed-41d2-b3ef-ec9065227e2b None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.252137 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841cd99-55ed-41d2-b3ef-ec9065227e2b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.252438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841cd99-55ed-41d2-b3ef-ec9065227e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.290555 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-3841cd99-55ed-41d2-b3ef-ec9065227e2b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:38.292598 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3841cd99-55ed-41d2-b3ef-ec9065227e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.451268 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1090/2181] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:38.473895 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979981a3-be10-49d7-927f-f9cdf8a328bb admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.475978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-979981a3-be10-49d7-927f-f9cdf8a328bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.552076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841cd99-55ed-41d2-b3ef-ec9065227e2b None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.554719 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3841cd99-55ed-41d2-b3ef-ec9065227e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.560100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979981a3-be10-49d7-927f-f9cdf8a328bb admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.566348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388c4dda-6310-4d44-9af1-f16a01dfe823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.567568 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c4dda-6310-4d44-9af1-f16a01dfe823 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.568120 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c4dda-6310-4d44-9af1-f16a01dfe823 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.568305 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c4dda-6310-4d44-9af1-f16a01dfe823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.580385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841cd99-55ed-41d2-b3ef-ec9065227e2b None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.581188 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c4dda-6310-4d44-9af1-f16a01dfe823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.583164 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388c4dda-6310-4d44-9af1-f16a01dfe823 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.591107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0578d36-fb78-48fa-9049-583f7f9d33a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.592103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0578d36-fb78-48fa-9049-583f7f9d33a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.592449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0578d36-fb78-48fa-9049-583f7f9d33a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.592785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0578d36-fb78-48fa-9049-583f7f9d33a0 admin admin] PATH_INFO: `/v3/projects/cdf8964872b8474ba0c96e0117879fdb` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.594116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0578d36-fb78-48fa-9049-583f7f9d33a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdf8964872b8474ba0c96e0117879fdb)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.596523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0578d36-fb78-48fa-9049-583f7f9d33a0 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.600213 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1092/2182] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 14:59:38.698621 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c0578d36-fb78-48fa-9049-583f7f9d33a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdf8964872b8474ba0c96e0117879fdb', 'request_id': 'req-c0578d36-fb78-48fa-9049-583f7f9d33a0'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.711624 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:38.725384 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c0578d36-fb78-48fa-9049-583f7f9d33a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdf8964872b8474ba0c96e0117879fdb', 'request_id': 'req-c0578d36-fb78-48fa-9049-583f7f9d33a0'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.737966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c0578d36-fb78-48fa-9049-583f7f9d33a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdf8964872b8474ba0c96e0117879fdb', 'request_id': 'req-c0578d36-fb78-48fa-9049-583f7f9d33a0'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.752006 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c0578d36-fb78-48fa-9049-583f7f9d33a0 admin admin] The token cache is being invalidate because project cdf8964872b8474ba0c96e0117879fdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:38.752310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c0578d36-fb78-48fa-9049-583f7f9d33a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.753008 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c0578d36-fb78-48fa-9049-583f7f9d33a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:38.758851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1091/2183] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:38 2022] DELETE /identity/v3/projects/cdf8964872b8474ba0c96e0117879fdb => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:38.759814 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24743319-6c1b-4b1c-bebe-d4fcd1f8bc27 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.760101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24743319-6c1b-4b1c-bebe-d4fcd1f8bc27 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.760376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24743319-6c1b-4b1c-bebe-d4fcd1f8bc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.781973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-24743319-6c1b-4b1c-bebe-d4fcd1f8bc27 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:38.781973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-24743319-6c1b-4b1c-bebe-d4fcd1f8bc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.801990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:38.880172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1093/2184] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:38.881801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd940b-7d4e-47dc-9a35-418ea44eaad5 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.882278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd940b-7d4e-47dc-9a35-418ea44eaad5 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.882373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd940b-7d4e-47dc-9a35-418ea44eaad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.907324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-22cd940b-7d4e-47dc-9a35-418ea44eaad5 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:38.909144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-22cd940b-7d4e-47dc-9a35-418ea44eaad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:38.925491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24743319-6c1b-4b1c-bebe-d4fcd1f8bc27 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.936709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24743319-6c1b-4b1c-bebe-d4fcd1f8bc27 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:38.937301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1092/2185] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:38.943725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38ac14-6b6f-4d94-bc41-c54610f025e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:38.944411 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38ac14-6b6f-4d94-bc41-c54610f025e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:38.944886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38ac14-6b6f-4d94-bc41-c54610f025e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:38.945309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38ac14-6b6f-4d94-bc41-c54610f025e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:38.946170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38ac14-6b6f-4d94-bc41-c54610f025e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:38.946629 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38ac14-6b6f-4d94-bc41-c54610f025e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:38.948615 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38ac14-6b6f-4d94-bc41-c54610f025e1 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:38.972516 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd940b-7d4e-47dc-9a35-418ea44eaad5 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:38.980290 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1094/2186] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:39.000893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd940b-7d4e-47dc-9a35-418ea44eaad5 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:39.007614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70669c-cce6-402f-b390-be5102373b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:39.008300 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70669c-cce6-402f-b390-be5102373b8d admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.008569 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70669c-cce6-402f-b390-be5102373b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.008899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70669c-cce6-402f-b390-be5102373b8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.010140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70669c-cce6-402f-b390-be5102373b8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:39.011530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70669c-cce6-402f-b390-be5102373b8d admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:39.092661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1093/2187] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:38 2022] POST /identity/v3/projects => generated 397 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:39.096878 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38ac14-6b6f-4d94-bc41-c54610f025e1 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:39.108411 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38ac14-6b6f-4d94-bc41-c54610f025e1 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:39.167462 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70669c-cce6-402f-b390-be5102373b8d admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:39.169429 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ff70669c-cce6-402f-b390-be5102373b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:39.247365 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003865d7-d4c9-4a88-b657-c16b6ca9723c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:39.248048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003865d7-d4c9-4a88-b657-c16b6ca9723c admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.251117 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003865d7-d4c9-4a88-b657-c16b6ca9723c admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.251399 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003865d7-d4c9-4a88-b657-c16b6ca9723c admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.252777 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003865d7-d4c9-4a88-b657-c16b6ca9723c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:39.257860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003865d7-d4c9-4a88-b657-c16b6ca9723c admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:39.300200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1095/2188] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:39 2022] POST /identity/v3/users => generated 299 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:39.302869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003865d7-d4c9-4a88-b657-c16b6ca9723c admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:39.332263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70669c-cce6-402f-b390-be5102373b8d admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:39.350602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e398779-032b-4eae-95cf-06d699c962b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:39.356582 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003865d7-d4c9-4a88-b657-c16b6ca9723c admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:39.356833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e398779-032b-4eae-95cf-06d699c962b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.357126 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e398779-032b-4eae-95cf-06d699c962b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.357396 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e398779-032b-4eae-95cf-06d699c962b5 admin admin] PATH_INFO: `/v3/projects/9d525d62168944648c5d72cb2cc4f8e2` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.361079 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e398779-032b-4eae-95cf-06d699c962b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d525d62168944648c5d72cb2cc4f8e2)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:39.363051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e398779-032b-4eae-95cf-06d699c962b5 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:39.375152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd4c97-2e3f-4486-97b1-3a4ee64afb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:39.376085 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd4c97-2e3f-4486-97b1-3a4ee64afb84 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.376213 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd4c97-2e3f-4486-97b1-3a4ee64afb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.376413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd4c97-2e3f-4486-97b1-3a4ee64afb84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.377072 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd4c97-2e3f-4486-97b1-3a4ee64afb84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:39.382872 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd4c97-2e3f-4486-97b1-3a4ee64afb84 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:39.407896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1096/2189] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:39.412386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd4c97-2e3f-4486-97b1-3a4ee64afb84 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:39.427532 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd4c97-2e3f-4486-97b1-3a4ee64afb84 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:39.436342 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6c2e3-c34e-45fd-8a28-fe106116777a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:39.437005 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6c2e3-c34e-45fd-8a28-fe106116777a admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.437343 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6c2e3-c34e-45fd-8a28-fe106116777a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.437649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6c2e3-c34e-45fd-8a28-fe106116777a admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.439046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6c2e3-c34e-45fd-8a28-fe106116777a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:39.441501 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6c2e3-c34e-45fd-8a28-fe106116777a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:39.465393 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1097/2190] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:39.470134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6c2e3-c34e-45fd-8a28-fe106116777a admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:39.480089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2e398779-032b-4eae-95cf-06d699c962b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d525d62168944648c5d72cb2cc4f8e2', 'request_id': 'req-2e398779-032b-4eae-95cf-06d699c962b5'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:39.490339 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6c2e3-c34e-45fd-8a28-fe106116777a admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:39.497899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2e398779-032b-4eae-95cf-06d699c962b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d525d62168944648c5d72cb2cc4f8e2', 'request_id': 'req-2e398779-032b-4eae-95cf-06d699c962b5'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:39.510401 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2e398779-032b-4eae-95cf-06d699c962b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d525d62168944648c5d72cb2cc4f8e2', 'request_id': 'req-2e398779-032b-4eae-95cf-06d699c962b5'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:39.532531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2e398779-032b-4eae-95cf-06d699c962b5 admin admin] The token cache is being invalidate because project 9d525d62168944648c5d72cb2cc4f8e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:39.532989 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2e398779-032b-4eae-95cf-06d699c962b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:39.535103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2e398779-032b-4eae-95cf-06d699c962b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:39.537115 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1094/2191] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:39 2022] DELETE /identity/v3/projects/9d525d62168944648c5d72cb2cc4f8e2 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:39.546706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0a0ea-56d7-4f5e-858f-0003a34d5892 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.547171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0a0ea-56d7-4f5e-858f-0003a34d5892 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.547729 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0a0ea-56d7-4f5e-858f-0003a34d5892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.604731 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-3aa0a0ea-56d7-4f5e-858f-0003a34d5892 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:39.606042 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3aa0a0ea-56d7-4f5e-858f-0003a34d5892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:39.634921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36a11c-0f97-4fdc-a056-74e2c9097ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:39.636376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36a11c-0f97-4fdc-a056-74e2c9097ead admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.636762 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36a11c-0f97-4fdc-a056-74e2c9097ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.637159 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36a11c-0f97-4fdc-a056-74e2c9097ead admin admin] PATH_INFO: `/v3/projects/feab6b1a63b041b68b6f689f68dafab0/users/5f91d0ff44cc47d79bd81ec9f93d39f1/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.663630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36a11c-0f97-4fdc-a056-74e2c9097ead admin admin] RBAC: Authorizing `identity:create_grant(project_id=feab6b1a63b041b68b6f689f68dafab0, user_id=5f91d0ff44cc47d79bd81ec9f93d39f1, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:39.666351 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:39.666828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36a11c-0f97-4fdc-a056-74e2c9097ead admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:39.688144 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1098/2192] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:39 2022] PUT /identity/v3/projects/feab6b1a63b041b68b6f689f68dafab0/users/5f91d0ff44cc47d79bd81ec9f93d39f1/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 14:59:39.707091 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06269c5-a693-4904-93fb-7a6fb6e21b22 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.707091 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06269c5-a693-4904-93fb-7a6fb6e21b22 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.707091 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06269c5-a693-4904-93fb-7a6fb6e21b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.719331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:39.722913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1095/2193] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:39.731126 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1317d-860a-4a4a-9f25-3d66e7e9be82 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.731282 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1317d-860a-4a4a-9f25-3d66e7e9be82 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.732016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1317d-860a-4a4a-9f25-3d66e7e9be82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.770420 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-8cc1317d-860a-4a4a-9f25-3d66e7e9be82 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:39.777599 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-8cc1317d-860a-4a4a-9f25-3d66e7e9be82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:39.800240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-d06269c5-a693-4904-93fb-7a6fb6e21b22 None None] MFA Rules not processed for user `5f91d0ff44cc47d79bd81ec9f93d39f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:39.811842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d06269c5-a693-4904-93fb-7a6fb6e21b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:39.922780 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1096/2194] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:39.924689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1317d-860a-4a4a-9f25-3d66e7e9be82 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:39.937261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1317d-860a-4a4a-9f25-3d66e7e9be82 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:39.944478 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af654fa0-3f3e-4208-93aa-4fec9b9076c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:39.945199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af654fa0-3f3e-4208-93aa-4fec9b9076c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:39.946030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af654fa0-3f3e-4208-93aa-4fec9b9076c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:39.946424 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af654fa0-3f3e-4208-93aa-4fec9b9076c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:39.947160 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af654fa0-3f3e-4208-93aa-4fec9b9076c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:39.947467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af654fa0-3f3e-4208-93aa-4fec9b9076c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:39.950908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1099/2195] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:39 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:39.951790 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af654fa0-3f3e-4208-93aa-4fec9b9076c9 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:39.962786 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1097/2196] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:39.967451 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06269c5-a693-4904-93fb-7a6fb6e21b22 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:39.975960 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af654fa0-3f3e-4208-93aa-4fec9b9076c9 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:39.978302 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-af654fa0-3f3e-4208-93aa-4fec9b9076c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:39.983438 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06269c5-a693-4904-93fb-7a6fb6e21b22 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:40.014983 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43641f5-577d-44c9-b091-dce0f13452c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:40.015741 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43641f5-577d-44c9-b091-dce0f13452c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.016118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43641f5-577d-44c9-b091-dce0f13452c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.016489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43641f5-577d-44c9-b091-dce0f13452c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.017414 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43641f5-577d-44c9-b091-dce0f13452c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:40.019640 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43641f5-577d-44c9-b091-dce0f13452c3 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:40.060890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1100/2197] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:39 2022] POST /identity/v3/projects => generated 389 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:40.067541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43641f5-577d-44c9-b091-dce0f13452c3 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:40.107256 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af654fa0-3f3e-4208-93aa-4fec9b9076c9 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:40.195869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43641f5-577d-44c9-b091-dce0f13452c3 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:40.197690 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c4bba-3e01-42d1-a7c7-014a64886340 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:40.198541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c4bba-3e01-42d1-a7c7-014a64886340 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.199050 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c4bba-3e01-42d1-a7c7-014a64886340 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.199522 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c4bba-3e01-42d1-a7c7-014a64886340 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.200951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4affa-36b6-4c5d-9238-db1601eb7049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:40.201562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4affa-36b6-4c5d-9238-db1601eb7049 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.201838 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4affa-36b6-4c5d-9238-db1601eb7049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.202106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4affa-36b6-4c5d-9238-db1601eb7049 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.202947 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4affa-36b6-4c5d-9238-db1601eb7049 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:40.205083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4affa-36b6-4c5d-9238-db1601eb7049 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:40.235071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1101/2198] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:40 2022] POST /identity/v3/users => generated 295 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 14:59:40.239615 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4affa-36b6-4c5d-9238-db1601eb7049 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:40.252971 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:40.253862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4affa-36b6-4c5d-9238-db1601eb7049 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:40.259329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1af39d-ffdb-44b3-a716-03603d66289a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:40.260081 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1af39d-ffdb-44b3-a716-03603d66289a admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.260364 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1af39d-ffdb-44b3-a716-03603d66289a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.260645 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1af39d-ffdb-44b3-a716-03603d66289a admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.261309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1af39d-ffdb-44b3-a716-03603d66289a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:40.263095 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1af39d-ffdb-44b3-a716-03603d66289a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:40.297919 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1102/2199] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:40.304099 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c4bba-3e01-42d1-a7c7-014a64886340 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:40.307580 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1af39d-ffdb-44b3-a716-03603d66289a admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:40.309413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c4bba-3e01-42d1-a7c7-014a64886340 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:40.331850 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1af39d-ffdb-44b3-a716-03603d66289a admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:40.339785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1098/2200] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:39 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:40.340599 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c4bba-3e01-42d1-a7c7-014a64886340 service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:40.344881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-5a0c4bba-3e01-42d1-a7c7-014a64886340 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:40.345951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578ae8b-d396-4221-8027-0a3e0cda6a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:40.346509 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578ae8b-d396-4221-8027-0a3e0cda6a96 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.346852 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578ae8b-d396-4221-8027-0a3e0cda6a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.347078 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578ae8b-d396-4221-8027-0a3e0cda6a96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.347808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578ae8b-d396-4221-8027-0a3e0cda6a96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:40.349502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578ae8b-d396-4221-8027-0a3e0cda6a96 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:40.374806 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1103/2201] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:40.379480 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578ae8b-d396-4221-8027-0a3e0cda6a96 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:40.379633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c4bba-3e01-42d1-a7c7-014a64886340 service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:40.392291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578ae8b-d396-4221-8027-0a3e0cda6a96 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:40.393448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f76c1-1772-4e09-b2b9-0ef1e75f8db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:40.394108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f76c1-1772-4e09-b2b9-0ef1e75f8db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.394390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f76c1-1772-4e09-b2b9-0ef1e75f8db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.394681 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f76c1-1772-4e09-b2b9-0ef1e75f8db3 admin admin] PATH_INFO: `/v3/users/0addaa15e4c844a5acd84c3a5e3869ff` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.397263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f76c1-1772-4e09-b2b9-0ef1e75f8db3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0addaa15e4c844a5acd84c3a5e3869ff)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:40.399492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3942b8-793f-4c23-b61e-9e7b3f0a4879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:40.399642 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f76c1-1772-4e09-b2b9-0ef1e75f8db3 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:40.400270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3942b8-793f-4c23-b61e-9e7b3f0a4879 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.400622 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3942b8-793f-4c23-b61e-9e7b3f0a4879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.400972 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3942b8-793f-4c23-b61e-9e7b3f0a4879 admin admin] PATH_INFO: `/v3/projects/c7088e4b6e684900a2da8e9c39ef3d72/users/748079e2495e4fd18e4a37da48d2ed42/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.421810 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3942b8-793f-4c23-b61e-9e7b3f0a4879 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7088e4b6e684900a2da8e9c39ef3d72, user_id=748079e2495e4fd18e4a37da48d2ed42, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:40.425298 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3942b8-793f-4c23-b61e-9e7b3f0a4879 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:40.442524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1104/2202] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:40 2022] PUT /identity/v3/projects/c7088e4b6e684900a2da8e9c39ef3d72/users/748079e2495e4fd18e4a37da48d2ed42/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:40.454442 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9fede-5fb6-4114-a08e-322ab55cd593 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.454901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9fede-5fb6-4114-a08e-322ab55cd593 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.455278 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9fede-5fb6-4114-a08e-322ab55cd593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.486342 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-fc8f76c1-1772-4e09-b2b9-0ef1e75f8db3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0addaa15e4c844a5acd84c3a5e3869ff'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:40.496015 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:40.498520 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-fc8f76c1-1772-4e09-b2b9-0ef1e75f8db3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0addaa15e4c844a5acd84c3a5e3869ff'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:40.500159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-fc8f76c1-1772-4e09-b2b9-0ef1e75f8db3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0addaa15e4c844a5acd84c3a5e3869ff'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:40.500756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-97e9fede-5fb6-4114-a08e-322ab55cd593 None None] MFA Rules not processed for user `748079e2495e4fd18e4a37da48d2ed42`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:40.503242 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-97e9fede-5fb6-4114-a08e-322ab55cd593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:40.522890 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-fc8f76c1-1772-4e09-b2b9-0ef1e75f8db3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0addaa15e4c844a5acd84c3a5e3869ff'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:40.524473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-fc8f76c1-1772-4e09-b2b9-0ef1e75f8db3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0addaa15e4c844a5acd84c3a5e3869ff'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:40.542666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1099/2203] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:40 2022] DELETE /identity/v3/users/0addaa15e4c844a5acd84c3a5e3869ff => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:40.612660 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1105/2204] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:40.623285 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f76c1-1772-4e09-b2b9-0ef1e75f8db3 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:40.624783 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-fc8f76c1-1772-4e09-b2b9-0ef1e75f8db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:40.699261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f76c1-1772-4e09-b2b9-0ef1e75f8db3 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:40.703907 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11819f3b-7a12-4767-a9fd-76cc8d8cc6b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:40.704438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11819f3b-7a12-4767-a9fd-76cc8d8cc6b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:40.704782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11819f3b-7a12-4767-a9fd-76cc8d8cc6b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:40.705121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11819f3b-7a12-4767-a9fd-76cc8d8cc6b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:40.777434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11819f3b-7a12-4767-a9fd-76cc8d8cc6b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:40.779214 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11819f3b-7a12-4767-a9fd-76cc8d8cc6b4 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:40.790212 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1100/2205] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:40.845550 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9fede-5fb6-4114-a08e-322ab55cd593 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:40.848112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-97e9fede-5fb6-4114-a08e-322ab55cd593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:40.952164 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9fede-5fb6-4114-a08e-322ab55cd593 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:41.016818 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea65319-5b06-4c69-9843-fdc4395b1471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:41.017749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea65319-5b06-4c69-9843-fdc4395b1471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:41.018135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea65319-5b06-4c69-9843-fdc4395b1471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:41.018530 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea65319-5b06-4c69-9843-fdc4395b1471 admin admin] PATH_INFO: `/v3/projects/d3cda271ef3244f391962778ed33c351` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:41.020179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea65319-5b06-4c69-9843-fdc4395b1471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3cda271ef3244f391962778ed33c351)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:41.024142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea65319-5b06-4c69-9843-fdc4395b1471 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:41.090936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-9ea65319-5b06-4c69-9843-fdc4395b1471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3cda271ef3244f391962778ed33c351', 'request_id': 'req-9ea65319-5b06-4c69-9843-fdc4395b1471'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:41.094958 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:41.107362 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-9ea65319-5b06-4c69-9843-fdc4395b1471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3cda271ef3244f391962778ed33c351', 'request_id': 'req-9ea65319-5b06-4c69-9843-fdc4395b1471'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:41.115074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-9ea65319-5b06-4c69-9843-fdc4395b1471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3cda271ef3244f391962778ed33c351', 'request_id': 'req-9ea65319-5b06-4c69-9843-fdc4395b1471'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:41.155784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-9ea65319-5b06-4c69-9843-fdc4395b1471 admin admin] The token cache is being invalidate because project d3cda271ef3244f391962778ed33c351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:41.157594 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-9ea65319-5b06-4c69-9843-fdc4395b1471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:41.158281 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-9ea65319-5b06-4c69-9843-fdc4395b1471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:41.160047 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1106/2206] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:40 2022] DELETE /identity/v3/projects/d3cda271ef3244f391962778ed33c351 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:41.166701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27c2f9-4403-496a-a756-3642c6676143 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:41.167135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27c2f9-4403-496a-a756-3642c6676143 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:41.167282 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27c2f9-4403-496a-a756-3642c6676143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:41.187253 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:41.199586 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-ac27c2f9-4403-496a-a756-3642c6676143 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:41.203417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ac27c2f9-4403-496a-a756-3642c6676143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:41.277469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1101/2207] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:41.285756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bc162-689e-4aa7-b33a-b05cc4d82137 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:41.286040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bc162-689e-4aa7-b33a-b05cc4d82137 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:41.286366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725bc162-689e-4aa7-b33a-b05cc4d82137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:41.310980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-725bc162-689e-4aa7-b33a-b05cc4d82137 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:41.310980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-725bc162-689e-4aa7-b33a-b05cc4d82137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:41.317532 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1107/2208] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 14:59:41.322238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27c2f9-4403-496a-a756-3642c6676143 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:41.332714 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27c2f9-4403-496a-a756-3642c6676143 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:41.337747 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024be689-7261-49ee-bd1f-bd76467923fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:41.338365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024be689-7261-49ee-bd1f-bd76467923fc admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:41.338646 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024be689-7261-49ee-bd1f-bd76467923fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:41.339000 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024be689-7261-49ee-bd1f-bd76467923fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:41.339936 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024be689-7261-49ee-bd1f-bd76467923fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 14:59:41.340163 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024be689-7261-49ee-bd1f-bd76467923fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:41.342055 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024be689-7261-49ee-bd1f-bd76467923fc admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:41.354392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1102/2209] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 14:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 14:59:41.360071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bc162-689e-4aa7-b33a-b05cc4d82137 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:41.370331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725bc162-689e-4aa7-b33a-b05cc4d82137 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:41.375087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183339ba-ab60-48b6-8437-75fd01e4bb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:41.375694 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183339ba-ab60-48b6-8437-75fd01e4bb0b admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:41.375971 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183339ba-ab60-48b6-8437-75fd01e4bb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:41.376240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183339ba-ab60-48b6-8437-75fd01e4bb0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:41.376882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183339ba-ab60-48b6-8437-75fd01e4bb0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:41.378804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183339ba-ab60-48b6-8437-75fd01e4bb0b admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:41.407323 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1108/2210] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 14:59:41 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:41.411363 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024be689-7261-49ee-bd1f-bd76467923fc admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:41.421341 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024be689-7261-49ee-bd1f-bd76467923fc admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:41.477706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3af337-a0b7-4842-b86b-6090fac59c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:41.478316 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3af337-a0b7-4842-b86b-6090fac59c9a admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:41.478594 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3af337-a0b7-4842-b86b-6090fac59c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:41.478895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3af337-a0b7-4842-b86b-6090fac59c9a admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:41.479539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3af337-a0b7-4842-b86b-6090fac59c9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:41.481454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3af337-a0b7-4842-b86b-6090fac59c9a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:41.504010 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1103/2211] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 14:59:41 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 14:59:41.507723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183339ba-ab60-48b6-8437-75fd01e4bb0b admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:41.520478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183339ba-ab60-48b6-8437-75fd01e4bb0b admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:41.525733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68390a0-6e38-47c2-beb5-0e2de3fff0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:41.526352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68390a0-6e38-47c2-beb5-0e2de3fff0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:41.526624 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68390a0-6e38-47c2-beb5-0e2de3fff0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:41.527072 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68390a0-6e38-47c2-beb5-0e2de3fff0b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:41.527755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68390a0-6e38-47c2-beb5-0e2de3fff0b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:41.529365 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68390a0-6e38-47c2-beb5-0e2de3fff0b7 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:41.542767 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1109/2212] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:41.544180 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3af337-a0b7-4842-b86b-6090fac59c9a admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:41.556357 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3af337-a0b7-4842-b86b-6090fac59c9a admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:41.562005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacb15a-e644-41ca-a267-b4475a2208c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:41.562634 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacb15a-e644-41ca-a267-b4475a2208c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:41.562984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacb15a-e644-41ca-a267-b4475a2208c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:41.563259 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacb15a-e644-41ca-a267-b4475a2208c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:41.563958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacb15a-e644-41ca-a267-b4475a2208c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:41.565546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacb15a-e644-41ca-a267-b4475a2208c7 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:41.577427 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1104/2213] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 14:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 14:59:41.581872 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68390a0-6e38-47c2-beb5-0e2de3fff0b7 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:41.593237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68390a0-6e38-47c2-beb5-0e2de3fff0b7 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:41.598289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b703d-6a04-46f9-9c07-ffc934f87929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:41.598962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b703d-6a04-46f9-9c07-ffc934f87929 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:41.599236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b703d-6a04-46f9-9c07-ffc934f87929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:41.599502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b703d-6a04-46f9-9c07-ffc934f87929 admin admin] PATH_INFO: `/v3/projects/1c7d5abbf8274516bf79312e3fa22f6c/users/da004a191bcd43b6a0865919813ec1a8/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:41.614855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b703d-6a04-46f9-9c07-ffc934f87929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7d5abbf8274516bf79312e3fa22f6c, user_id=da004a191bcd43b6a0865919813ec1a8, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:41.618158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b703d-6a04-46f9-9c07-ffc934f87929 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:41.631030 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1110/2214] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 14:59:41 2022] PUT /identity/v3/projects/1c7d5abbf8274516bf79312e3fa22f6c/users/da004a191bcd43b6a0865919813ec1a8/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:41.636249 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a35ac-57e9-4986-928c-947d7ca97596 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:41.636554 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a35ac-57e9-4986-928c-947d7ca97596 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:41.636909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a35ac-57e9-4986-928c-947d7ca97596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:41.656934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-4a1a35ac-57e9-4986-928c-947d7ca97596 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:41.658166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-4a1a35ac-57e9-4986-928c-947d7ca97596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:41.717562 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1105/2215] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:41.727689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b703d-6a04-46f9-9c07-ffc934f87929 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:41.729362 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-715b703d-6a04-46f9-9c07-ffc934f87929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:41.785602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b703d-6a04-46f9-9c07-ffc934f87929 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:41.789636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f077631-302f-43f7-98d1-081e607adc08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:41.790235 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f077631-302f-43f7-98d1-081e607adc08 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:41.790505 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f077631-302f-43f7-98d1-081e607adc08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:41.790797 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f077631-302f-43f7-98d1-081e607adc08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:41.799619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f077631-302f-43f7-98d1-081e607adc08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:41.801062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f077631-302f-43f7-98d1-081e607adc08 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:41.811142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1111/2216] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:46.495250 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f0c43-9d7d-4716-9610-0a617da97982 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:46.495782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f0c43-9d7d-4716-9610-0a617da97982 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:46.495782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f0c43-9d7d-4716-9610-0a617da97982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:46.545813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:46.554370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-d58f0c43-9d7d-4716-9610-0a617da97982 None None] MFA Rules not processed for user `da004a191bcd43b6a0865919813ec1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:46.556241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d58f0c43-9d7d-4716-9610-0a617da97982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:46.627493 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1106/2217] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:46.642631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f077631-302f-43f7-98d1-081e607adc08 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:46.644308 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-6f077631-302f-43f7-98d1-081e607adc08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:46.738192 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f077631-302f-43f7-98d1-081e607adc08 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:46.744329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c630606-c768-4ec5-a872-0675e15ed013 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:46.744943 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c630606-c768-4ec5-a872-0675e15ed013 service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:46.745223 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c630606-c768-4ec5-a872-0675e15ed013 service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:46.745494 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c630606-c768-4ec5-a872-0675e15ed013 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:46.753763 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c630606-c768-4ec5-a872-0675e15ed013 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:46.764032 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c630606-c768-4ec5-a872-0675e15ed013 service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:46.773995 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1112/2218] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:48.363879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb637e-f51e-4b8b-892a-c160606c166d None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:48.364429 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb637e-f51e-4b8b-892a-c160606c166d None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:48.364429 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb637e-f51e-4b8b-892a-c160606c166d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:48.384828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-41fb637e-f51e-4b8b-892a-c160606c166d None None] MFA Rules not processed for user `da004a191bcd43b6a0865919813ec1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:48.386037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-41fb637e-f51e-4b8b-892a-c160606c166d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:48.393544 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1107/2219] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:52.989953 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c630606-c768-4ec5-a872-0675e15ed013 service nova] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:52.991469 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-8c630606-c768-4ec5-a872-0675e15ed013 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:53.004848 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 14:59:53.009748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c630606-c768-4ec5-a872-0675e15ed013 service nova] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:53.015930 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d42f53-c3de-4e8c-9db2-3e59ad5a973c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:53.016634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d42f53-c3de-4e8c-9db2-3e59ad5a973c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:53.017110 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d42f53-c3de-4e8c-9db2-3e59ad5a973c admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:53.017560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d42f53-c3de-4e8c-9db2-3e59ad5a973c admin admin] PATH_INFO: `/v3/users/5f91d0ff44cc47d79bd81ec9f93d39f1` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:53.018888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d42f53-c3de-4e8c-9db2-3e59ad5a973c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f91d0ff44cc47d79bd81ec9f93d39f1)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:53.020852 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d42f53-c3de-4e8c-9db2-3e59ad5a973c admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:53.069017 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a7d42f53-c3de-4e8c-9db2-3e59ad5a973c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f91d0ff44cc47d79bd81ec9f93d39f1'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.076389 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a7d42f53-c3de-4e8c-9db2-3e59ad5a973c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f91d0ff44cc47d79bd81ec9f93d39f1'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.083578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a7d42f53-c3de-4e8c-9db2-3e59ad5a973c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f91d0ff44cc47d79bd81ec9f93d39f1'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.095676 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a7d42f53-c3de-4e8c-9db2-3e59ad5a973c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f91d0ff44cc47d79bd81ec9f93d39f1'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.096337 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a7d42f53-c3de-4e8c-9db2-3e59ad5a973c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f91d0ff44cc47d79bd81ec9f93d39f1'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.102689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1113/2220] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 14:59:52 2022] DELETE /identity/v3/users/5f91d0ff44cc47d79bd81ec9f93d39f1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:53.107234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea995816-89dd-49fc-bc51-3d4eb831cb08 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:53.107705 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea995816-89dd-49fc-bc51-3d4eb831cb08 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:53.107795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea995816-89dd-49fc-bc51-3d4eb831cb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:53.127619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-ea995816-89dd-49fc-bc51-3d4eb831cb08 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:53.128867 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ea995816-89dd-49fc-bc51-3d4eb831cb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:53.188734 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1108/2221] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 14:59:53.199057 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d42f53-c3de-4e8c-9db2-3e59ad5a973c admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:53.201270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a7d42f53-c3de-4e8c-9db2-3e59ad5a973c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:53.284098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d42f53-c3de-4e8c-9db2-3e59ad5a973c admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:53.288374 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8f79f-d45d-44d9-b37d-5c755be07cdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:53.289976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8f79f-d45d-44d9-b37d-5c755be07cdf service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:53.290258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8f79f-d45d-44d9-b37d-5c755be07cdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:53.290531 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8f79f-d45d-44d9-b37d-5c755be07cdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:53.307595 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f8f79f-d45d-44d9-b37d-5c755be07cdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:53.310233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f8f79f-d45d-44d9-b37d-5c755be07cdf service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:53.348805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1114/2222] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 14:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 14:59:53.676972 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea995816-89dd-49fc-bc51-3d4eb831cb08 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 14:59:53.678655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ea995816-89dd-49fc-bc51-3d4eb831cb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:53.693791 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea995816-89dd-49fc-bc51-3d4eb831cb08 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 14:59:53.701094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2678e-42e7-494f-bb90-d95deb0fc769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 14:59:53.701779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2678e-42e7-494f-bb90-d95deb0fc769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:53.702064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2678e-42e7-494f-bb90-d95deb0fc769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:53.702337 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2678e-42e7-494f-bb90-d95deb0fc769 admin admin] PATH_INFO: `/v3/projects/feab6b1a63b041b68b6f689f68dafab0` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:53.703551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2678e-42e7-494f-bb90-d95deb0fc769 admin admin] RBAC: Authorizing `identity:delete_project(project_id=feab6b1a63b041b68b6f689f68dafab0)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 14:59:53.705741 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2678e-42e7-494f-bb90-d95deb0fc769 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 14:59:53.760088 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2eb2678e-42e7-494f-bb90-d95deb0fc769 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feab6b1a63b041b68b6f689f68dafab0', 'request_id': 'req-2eb2678e-42e7-494f-bb90-d95deb0fc769'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.774666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2eb2678e-42e7-494f-bb90-d95deb0fc769 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feab6b1a63b041b68b6f689f68dafab0', 'request_id': 'req-2eb2678e-42e7-494f-bb90-d95deb0fc769'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.783155 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2eb2678e-42e7-494f-bb90-d95deb0fc769 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feab6b1a63b041b68b6f689f68dafab0', 'request_id': 'req-2eb2678e-42e7-494f-bb90-d95deb0fc769'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.794459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2eb2678e-42e7-494f-bb90-d95deb0fc769 admin admin] The token cache is being invalidate because project feab6b1a63b041b68b6f689f68dafab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 14:59:53.794777 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2eb2678e-42e7-494f-bb90-d95deb0fc769 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.795437 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2eb2678e-42e7-494f-bb90-d95deb0fc769 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 14:59:53.798109 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1109/2223] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 14:59:53 2022] DELETE /identity/v3/projects/feab6b1a63b041b68b6f689f68dafab0 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 14:59:53.988223 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6dc76-a1af-469e-ae19-3707934bde18 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 14:59:53.988508 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6dc76-a1af-469e-ae19-3707934bde18 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 14:59:53.988793 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6dc76-a1af-469e-ae19-3707934bde18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 14:59:54.009076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-96d6dc76-a1af-469e-ae19-3707934bde18 None None] MFA Rules not processed for user `6465d1250d2146b2be8d17a70b8bb929`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 14:59:54.010291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-96d6dc76-a1af-469e-ae19-3707934bde18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 14:59:54.065623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1115/2224] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:02.868934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2678e-42e7-494f-bb90-d95deb0fc769 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:02.870511 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-2eb2678e-42e7-494f-bb90-d95deb0fc769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:02.894114 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:02.964677 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2678e-42e7-494f-bb90-d95deb0fc769 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:02.975435 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e32510-b88d-4a71-bb1a-4daa85400be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:02.976240 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e32510-b88d-4a71-bb1a-4daa85400be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:02.976591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e32510-b88d-4a71-bb1a-4daa85400be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:02.976938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e32510-b88d-4a71-bb1a-4daa85400be6 admin admin] PATH_INFO: `/v3/users/748079e2495e4fd18e4a37da48d2ed42` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:02.978758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e32510-b88d-4a71-bb1a-4daa85400be6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=748079e2495e4fd18e4a37da48d2ed42)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:02.980825 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e32510-b88d-4a71-bb1a-4daa85400be6 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:03.044413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-92e32510-b88d-4a71-bb1a-4daa85400be6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '748079e2495e4fd18e4a37da48d2ed42'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:03.056407 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-92e32510-b88d-4a71-bb1a-4daa85400be6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '748079e2495e4fd18e4a37da48d2ed42'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:03.057407 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-92e32510-b88d-4a71-bb1a-4daa85400be6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '748079e2495e4fd18e4a37da48d2ed42'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:03.072951 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-92e32510-b88d-4a71-bb1a-4daa85400be6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '748079e2495e4fd18e4a37da48d2ed42'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:03.074052 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-92e32510-b88d-4a71-bb1a-4daa85400be6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '748079e2495e4fd18e4a37da48d2ed42'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:03.094808 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1110/2225] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:02 2022] DELETE /identity/v3/users/748079e2495e4fd18e4a37da48d2ed42 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:03.096509 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22fb54-e7f6-466f-ad5e-e0ed60821184 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:03.096995 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22fb54-e7f6-466f-ad5e-e0ed60821184 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:03.097500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22fb54-e7f6-466f-ad5e-e0ed60821184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:03.119672 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-3a22fb54-e7f6-466f-ad5e-e0ed60821184 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:03.121097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3a22fb54-e7f6-466f-ad5e-e0ed60821184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:03.148007 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:03.197872 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1116/2226] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:03.209205 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e32510-b88d-4a71-bb1a-4daa85400be6 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:03.211474 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-92e32510-b88d-4a71-bb1a-4daa85400be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:03.299143 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e32510-b88d-4a71-bb1a-4daa85400be6 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:03.310618 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dffb6-4abf-4e37-9b0f-533090a8e6d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:03.310618 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dffb6-4abf-4e37-9b0f-533090a8e6d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:03.310618 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dffb6-4abf-4e37-9b0f-533090a8e6d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:03.310618 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dffb6-4abf-4e37-9b0f-533090a8e6d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:03.339711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2dffb6-4abf-4e37-9b0f-533090a8e6d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:03.341182 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2dffb6-4abf-4e37-9b0f-533090a8e6d1 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:03.374648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1111/2227] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:03.646010 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfcd4d-f25c-4040-9dcc-a57d3c6f24a7 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:03.646322 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfcd4d-f25c-4040-9dcc-a57d3c6f24a7 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:03.646617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfcd4d-f25c-4040-9dcc-a57d3c6f24a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:03.671678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-cabfcd4d-f25c-4040-9dcc-a57d3c6f24a7 None None] MFA Rules not processed for user `6256407d9df648d5bbe33495cb9a8b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:03.673534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-cabfcd4d-f25c-4040-9dcc-a57d3c6f24a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:03.785884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1117/2228] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Sep 22 15:00:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:03.842323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dffb6-4abf-4e37-9b0f-533090a8e6d1 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:03.847286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-5f2dffb6-4abf-4e37-9b0f-533090a8e6d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:03.871266 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfcd4d-f25c-4040-9dcc-a57d3c6f24a7 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:03.872626 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-cabfcd4d-f25c-4040-9dcc-a57d3c6f24a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:03.887731 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfcd4d-f25c-4040-9dcc-a57d3c6f24a7 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:03.893236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9982d68-24ae-4810-85aa-00ed71ed53ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:03.893771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9982d68-24ae-4810-85aa-00ed71ed53ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:03.894050 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9982d68-24ae-4810-85aa-00ed71ed53ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:03.894318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9982d68-24ae-4810-85aa-00ed71ed53ac admin admin] PATH_INFO: `/v3/projects/c7088e4b6e684900a2da8e9c39ef3d72` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:03.895473 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9982d68-24ae-4810-85aa-00ed71ed53ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7088e4b6e684900a2da8e9c39ef3d72)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:03.897443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9982d68-24ae-4810-85aa-00ed71ed53ac admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:03.941785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:04.007516 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dffb6-4abf-4e37-9b0f-533090a8e6d1 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:04.007841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f9982d68-24ae-4810-85aa-00ed71ed53ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7088e4b6e684900a2da8e9c39ef3d72', 'request_id': 'req-f9982d68-24ae-4810-85aa-00ed71ed53ac'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.017320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f9982d68-24ae-4810-85aa-00ed71ed53ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7088e4b6e684900a2da8e9c39ef3d72', 'request_id': 'req-f9982d68-24ae-4810-85aa-00ed71ed53ac'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.025851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f9982d68-24ae-4810-85aa-00ed71ed53ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7088e4b6e684900a2da8e9c39ef3d72', 'request_id': 'req-f9982d68-24ae-4810-85aa-00ed71ed53ac'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.043519 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f9982d68-24ae-4810-85aa-00ed71ed53ac admin admin] The token cache is being invalidate because project c7088e4b6e684900a2da8e9c39ef3d72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:04.043877 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f9982d68-24ae-4810-85aa-00ed71ed53ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.044638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f9982d68-24ae-4810-85aa-00ed71ed53ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:04.055302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1118/2229] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:03 2022] DELETE /identity/v3/projects/c7088e4b6e684900a2da8e9c39ef3d72 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:00:04.104602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947748a-6424-4f3c-b909-85fd1141f98a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a265623ae734936a340c97df852e227', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:04.106016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947748a-6424-4f3c-b909-85fd1141f98a service swift] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:04.106526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947748a-6424-4f3c-b909-85fd1141f98a service swift] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:04.107205 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947748a-6424-4f3c-b909-85fd1141f98a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:04.109464 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3947748a-6424-4f3c-b909-85fd1141f98a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:04.208642 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947748a-6424-4f3c-b909-85fd1141f98a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:04.211638 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947748a-6424-4f3c-b909-85fd1141f98a service swift] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:04.225592 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1112/2230] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Sep 22 15:00:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:00:04.825097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7493e-5763-4637-81d7-db20c3dc4f8a None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:04.825667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7493e-5763-4637-81d7-db20c3dc4f8a None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:04.825667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7493e-5763-4637-81d7-db20c3dc4f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:04.846276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-e9a7493e-5763-4637-81d7-db20c3dc4f8a None None] MFA Rules not processed for user `6256407d9df648d5bbe33495cb9a8b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:04.847535 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e9a7493e-5763-4637-81d7-db20c3dc4f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:04.854835 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1119/2231] 10.222.0.26 () {58 vars in 967 bytes} [Thu Sep 22 15:00:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:04.868239 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947748a-6424-4f3c-b909-85fd1141f98a service swift] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:04.869669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3947748a-6424-4f3c-b909-85fd1141f98a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:04.884192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947748a-6424-4f3c-b909-85fd1141f98a service swift] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:04.889244 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85b78d-df37-4dda-82cf-6f4537aa56e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a265623ae734936a340c97df852e227', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:04.889863 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85b78d-df37-4dda-82cf-6f4537aa56e1 service swift] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:04.890138 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85b78d-df37-4dda-82cf-6f4537aa56e1 service swift] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:04.890404 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85b78d-df37-4dda-82cf-6f4537aa56e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:04.899750 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85b78d-df37-4dda-82cf-6f4537aa56e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:04.901313 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85b78d-df37-4dda-82cf-6f4537aa56e1 service swift] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:04.909281 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1113/2232] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Sep 22 15:00:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:00:05.188129 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353876a7-d43c-4efa-bed7-0d7a387745de None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.188416 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353876a7-d43c-4efa-bed7-0d7a387745de None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.188790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353876a7-d43c-4efa-bed7-0d7a387745de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.208782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-353876a7-d43c-4efa-bed7-0d7a387745de None None] MFA Rules not processed for user `6256407d9df648d5bbe33495cb9a8b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:05.210718 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-353876a7-d43c-4efa-bed7-0d7a387745de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:05.218441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1120/2233] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Sep 22 15:00:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:05.231573 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85b78d-df37-4dda-82cf-6f4537aa56e1 service swift] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:05.243180 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85b78d-df37-4dda-82cf-6f4537aa56e1 service swift] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:05.248455 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fad04-725a-4fbb-af0b-9c8ca86881cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a265623ae734936a340c97df852e227', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:05.249067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fad04-725a-4fbb-af0b-9c8ca86881cb service swift] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.249343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fad04-725a-4fbb-af0b-9c8ca86881cb service swift] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.249614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fad04-725a-4fbb-af0b-9c8ca86881cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.258715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7fad04-725a-4fbb-af0b-9c8ca86881cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:05.260407 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7fad04-725a-4fbb-af0b-9c8ca86881cb service swift] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:05.269506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1114/2234] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Sep 22 15:00:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:00:05.620962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c93ebf-3921-4389-9340-7000af3b171d None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.621247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c93ebf-3921-4389-9340-7000af3b171d None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.621524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c93ebf-3921-4389-9340-7000af3b171d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.640860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-d5c93ebf-3921-4389-9340-7000af3b171d None None] MFA Rules not processed for user `6256407d9df648d5bbe33495cb9a8b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:05.641984 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d5c93ebf-3921-4389-9340-7000af3b171d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:05.649596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1121/2235] 10.222.0.26 () {58 vars in 967 bytes} [Thu Sep 22 15:00:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:05.662672 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fad04-725a-4fbb-af0b-9c8ca86881cb service swift] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:05.673148 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fad04-725a-4fbb-af0b-9c8ca86881cb service swift] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:05.678212 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb3fcf-efce-4f8f-873a-341c4b625977 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a265623ae734936a340c97df852e227', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:05.678852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb3fcf-efce-4f8f-873a-341c4b625977 service swift] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:05.679133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb3fcf-efce-4f8f-873a-341c4b625977 service swift] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:05.679408 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb3fcf-efce-4f8f-873a-341c4b625977 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:05.687888 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb3fcf-efce-4f8f-873a-341c4b625977 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:05.689597 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb3fcf-efce-4f8f-873a-341c4b625977 service swift] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:05.699158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1115/2236] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Sep 22 15:00:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:00:07.932246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c93ebf-3921-4389-9340-7000af3b171d None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:07.933658 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d5c93ebf-3921-4389-9340-7000af3b171d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:07.967823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:08.000950 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c93ebf-3921-4389-9340-7000af3b171d None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:08.005880 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d829665-9892-4a13-be39-5e417e3aa894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.006514 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d829665-9892-4a13-be39-5e417e3aa894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.006827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d829665-9892-4a13-be39-5e417e3aa894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.007103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d829665-9892-4a13-be39-5e417e3aa894 admin admin] PATH_INFO: `/v3/users/5b75e9af270641f8a96ed7a7be1e0c53` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.008244 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d829665-9892-4a13-be39-5e417e3aa894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b75e9af270641f8a96ed7a7be1e0c53)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.010163 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d829665-9892-4a13-be39-5e417e3aa894 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:08.057670 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-2d829665-9892-4a13-be39-5e417e3aa894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b75e9af270641f8a96ed7a7be1e0c53'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:08.064419 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-2d829665-9892-4a13-be39-5e417e3aa894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b75e9af270641f8a96ed7a7be1e0c53'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:08.071364 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-2d829665-9892-4a13-be39-5e417e3aa894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b75e9af270641f8a96ed7a7be1e0c53'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:08.083651 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-2d829665-9892-4a13-be39-5e417e3aa894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b75e9af270641f8a96ed7a7be1e0c53'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:08.084321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-2d829665-9892-4a13-be39-5e417e3aa894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b75e9af270641f8a96ed7a7be1e0c53'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:08.091611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1122/2237] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:07 2022] DELETE /identity/v3/users/5b75e9af270641f8a96ed7a7be1e0c53 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:08.096205 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc49c4a-5218-4a80-b68b-454cf9af276b None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.096716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc49c4a-5218-4a80-b68b-454cf9af276b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.096835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc49c4a-5218-4a80-b68b-454cf9af276b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.126187 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-7fc49c4a-5218-4a80-b68b-454cf9af276b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:08.127470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7fc49c4a-5218-4a80-b68b-454cf9af276b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:08.170159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:08.188705 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1116/2238] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:08.198808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d829665-9892-4a13-be39-5e417e3aa894 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:08.200224 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2d829665-9892-4a13-be39-5e417e3aa894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:08.261984 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d829665-9892-4a13-be39-5e417e3aa894 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:08.265690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fba52-4c69-4614-a492-7b38364fd696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.266326 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fba52-4c69-4614-a492-7b38364fd696 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.266748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fba52-4c69-4614-a492-7b38364fd696 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.267038 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fba52-4c69-4614-a492-7b38364fd696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.277363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fba52-4c69-4614-a492-7b38364fd696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.278890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fba52-4c69-4614-a492-7b38364fd696 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:08.293905 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1123/2239] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:08.463457 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc49c4a-5218-4a80-b68b-454cf9af276b None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:08.465103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7fc49c4a-5218-4a80-b68b-454cf9af276b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:08.476839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc49c4a-5218-4a80-b68b-454cf9af276b None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:08.481399 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df93846-8536-4c02-8f3f-044d42e9c09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.482027 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df93846-8536-4c02-8f3f-044d42e9c09d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.482298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df93846-8536-4c02-8f3f-044d42e9c09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.482566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df93846-8536-4c02-8f3f-044d42e9c09d admin admin] PATH_INFO: `/v3/projects/412a2bb8e4e24a4db88f4ce940bf6a34` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.483718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df93846-8536-4c02-8f3f-044d42e9c09d admin admin] RBAC: Authorizing `identity:delete_project(project_id=412a2bb8e4e24a4db88f4ce940bf6a34)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.486084 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df93846-8536-4c02-8f3f-044d42e9c09d admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:08.532503 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-1df93846-8536-4c02-8f3f-044d42e9c09d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '412a2bb8e4e24a4db88f4ce940bf6a34', 'request_id': 'req-1df93846-8536-4c02-8f3f-044d42e9c09d'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:08.539270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-1df93846-8536-4c02-8f3f-044d42e9c09d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '412a2bb8e4e24a4db88f4ce940bf6a34', 'request_id': 'req-1df93846-8536-4c02-8f3f-044d42e9c09d'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:08.545498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-1df93846-8536-4c02-8f3f-044d42e9c09d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '412a2bb8e4e24a4db88f4ce940bf6a34', 'request_id': 'req-1df93846-8536-4c02-8f3f-044d42e9c09d'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:08.556972 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-1df93846-8536-4c02-8f3f-044d42e9c09d admin admin] The token cache is being invalidate because project 412a2bb8e4e24a4db88f4ce940bf6a34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:08.557290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-1df93846-8536-4c02-8f3f-044d42e9c09d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:08.557918 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-1df93846-8536-4c02-8f3f-044d42e9c09d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:08.559469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1117/2240] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:08 2022] DELETE /identity/v3/projects/412a2bb8e4e24a4db88f4ce940bf6a34 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:08.564435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2912e-d38f-44ac-a03d-4eafae2eb1c4 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.564860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2912e-d38f-44ac-a03d-4eafae2eb1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.564961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2912e-d38f-44ac-a03d-4eafae2eb1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.585615 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-76c2912e-d38f-44ac-a03d-4eafae2eb1c4 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:08.586862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-76c2912e-d38f-44ac-a03d-4eafae2eb1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:08.642428 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:08.665717 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1124/2241] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:08.667569 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0b611-9ed1-4236-9535-70ceabcae0c7 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.668129 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0b611-9ed1-4236-9535-70ceabcae0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.668588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0b611-9ed1-4236-9535-70ceabcae0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.691285 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-a3c0b611-9ed1-4236-9535-70ceabcae0c7 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:08.692694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a3c0b611-9ed1-4236-9535-70ceabcae0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:08.707149 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1118/2242] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:08.708090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2912e-d38f-44ac-a03d-4eafae2eb1c4 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:08.719292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2912e-d38f-44ac-a03d-4eafae2eb1c4 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:08.724203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2984f-1192-417c-9226-4e396bfb7c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.724886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2984f-1192-417c-9226-4e396bfb7c41 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.725168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2984f-1192-417c-9226-4e396bfb7c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.725543 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2984f-1192-417c-9226-4e396bfb7c41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.726269 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f2984f-1192-417c-9226-4e396bfb7c41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:00:08.726692 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f2984f-1192-417c-9226-4e396bfb7c41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.728412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f2984f-1192-417c-9226-4e396bfb7c41 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:08.741127 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1125/2243] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 15:00:08.745337 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0b611-9ed1-4236-9535-70ceabcae0c7 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:08.754869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0b611-9ed1-4236-9535-70ceabcae0c7 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:08.760974 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c5ebc-7644-4ce0-8ca4-380588fc798a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.761623 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c5ebc-7644-4ce0-8ca4-380588fc798a admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.761876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c5ebc-7644-4ce0-8ca4-380588fc798a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.762200 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c5ebc-7644-4ce0-8ca4-380588fc798a admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.762859 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c5ebc-7644-4ce0-8ca4-380588fc798a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.764814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c5ebc-7644-4ce0-8ca4-380588fc798a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:08.799551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1119/2244] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:08 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:08.800927 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2984f-1192-417c-9226-4e396bfb7c41 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:08.811491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2984f-1192-417c-9226-4e396bfb7c41 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:08.883326 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa51a5-bd17-439f-ac65-a28185702bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.884048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa51a5-bd17-439f-ac65-a28185702bac admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.884399 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa51a5-bd17-439f-ac65-a28185702bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.884772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa51a5-bd17-439f-ac65-a28185702bac admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.885502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa51a5-bd17-439f-ac65-a28185702bac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.887869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa51a5-bd17-439f-ac65-a28185702bac admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:08.920353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1126/2245] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:08 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:08.922526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c5ebc-7644-4ce0-8ca4-380588fc798a admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:08.936572 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c5ebc-7644-4ce0-8ca4-380588fc798a admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:08.944364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef1d5f-7435-47c7-8ba9-c2b9bd9713fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.945291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef1d5f-7435-47c7-8ba9-c2b9bd9713fb admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.945959 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef1d5f-7435-47c7-8ba9-c2b9bd9713fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.946456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef1d5f-7435-47c7-8ba9-c2b9bd9713fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.947459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef1d5f-7435-47c7-8ba9-c2b9bd9713fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.949132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef1d5f-7435-47c7-8ba9-c2b9bd9713fb admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:08.966773 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1120/2246] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:08.971725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa51a5-bd17-439f-ac65-a28185702bac admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:08.984463 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa51a5-bd17-439f-ac65-a28185702bac admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:08.989606 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7add9d75-5b3d-469c-ab8b-a928ed3eaf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:08.990281 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add9d75-5b3d-469c-ab8b-a928ed3eaf7f admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:08.990699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add9d75-5b3d-469c-ab8b-a928ed3eaf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:08.990884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7add9d75-5b3d-469c-ab8b-a928ed3eaf7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:08.991514 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add9d75-5b3d-469c-ab8b-a928ed3eaf7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:08.993131 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7add9d75-5b3d-469c-ab8b-a928ed3eaf7f admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:09.004909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1127/2247] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:00:09.009278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef1d5f-7435-47c7-8ba9-c2b9bd9713fb admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:09.022882 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef1d5f-7435-47c7-8ba9-c2b9bd9713fb admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:09.028639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac9b1e-ecd9-4e4c-9b30-6f156d82ac35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:09.029528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac9b1e-ecd9-4e4c-9b30-6f156d82ac35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:09.029878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac9b1e-ecd9-4e4c-9b30-6f156d82ac35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:09.030218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac9b1e-ecd9-4e4c-9b30-6f156d82ac35 admin admin] PATH_INFO: `/v3/projects/26875c24ce944501a6b23188c784d9f5/users/8463811a630e4b90b1cd96e4dec020a2/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:09.048086 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac9b1e-ecd9-4e4c-9b30-6f156d82ac35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26875c24ce944501a6b23188c784d9f5, user_id=8463811a630e4b90b1cd96e4dec020a2, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:09.051757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac9b1e-ecd9-4e4c-9b30-6f156d82ac35 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:09.074709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1121/2248] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:09 2022] PUT /identity/v3/projects/26875c24ce944501a6b23188c784d9f5/users/8463811a630e4b90b1cd96e4dec020a2/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:09.076980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fe304-5d62-4610-9434-c1329707d496 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:09.077299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fe304-5d62-4610-9434-c1329707d496 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:09.077578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fe304-5d62-4610-9434-c1329707d496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:09.099138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-390fe304-5d62-4610-9434-c1329707d496 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:09.100976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-390fe304-5d62-4610-9434-c1329707d496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:09.174902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1128/2249] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:09.184501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac9b1e-ecd9-4e4c-9b30-6f156d82ac35 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:09.186895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-07ac9b1e-ecd9-4e4c-9b30-6f156d82ac35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:09.246089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac9b1e-ecd9-4e4c-9b30-6f156d82ac35 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:09.249808 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb27ed-a836-4418-8dbe-acd1007468a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:09.250425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb27ed-a836-4418-8dbe-acd1007468a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:09.250706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb27ed-a836-4418-8dbe-acd1007468a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:09.251012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb27ed-a836-4418-8dbe-acd1007468a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:09.260386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb27ed-a836-4418-8dbe-acd1007468a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:09.261805 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb27ed-a836-4418-8dbe-acd1007468a5 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:09.271250 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1122/2250] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:10.314860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0c780-89f6-482d-997a-256f6d7ed78f None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:10.315363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0c780-89f6-482d-997a-256f6d7ed78f None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:10.315479 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0c780-89f6-482d-997a-256f6d7ed78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:10.344414 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-4de0c780-89f6-482d-997a-256f6d7ed78f None None] MFA Rules not processed for user `8463811a630e4b90b1cd96e4dec020a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:10.345699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4de0c780-89f6-482d-997a-256f6d7ed78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:10.403229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:10.423507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1129/2251] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:00:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:10.434242 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb27ed-a836-4418-8dbe-acd1007468a5 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:10.435718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-32bb27ed-a836-4418-8dbe-acd1007468a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:10.447433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:10.498368 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb27ed-a836-4418-8dbe-acd1007468a5 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:10.502819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7c5c3-5895-494b-b8a2-014942f2a483 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:10.503374 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7c5c3-5895-494b-b8a2-014942f2a483 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:10.503648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7c5c3-5895-494b-b8a2-014942f2a483 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:10.503917 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7c5c3-5895-494b-b8a2-014942f2a483 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:10.513080 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7c5c3-5895-494b-b8a2-014942f2a483 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:10.514485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e7c5c3-5895-494b-b8a2-014942f2a483 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:10.524090 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1123/2252] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:00:11.925295 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ab9c4-44c8-4f3e-b645-f41848dc94c9 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:11.925824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ab9c4-44c8-4f3e-b645-f41848dc94c9 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:11.925915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ab9c4-44c8-4f3e-b645-f41848dc94c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:11.945326 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-958ab9c4-44c8-4f3e-b645-f41848dc94c9 None None] MFA Rules not processed for user `8463811a630e4b90b1cd96e4dec020a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:11.946523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-958ab9c4-44c8-4f3e-b645-f41848dc94c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:11.953338 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1130/2253] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:00:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:20.519265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=22 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:21.939266 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=7 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:24.154007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa0354-7818-4c85-b5d6-47c7f2f6e1fd None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:24.154540 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa0354-7818-4c85-b5d6-47c7f2f6e1fd None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:24.154650 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa0354-7818-4c85-b5d6-47c7f2f6e1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:24.176362 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-a8fa0354-7818-4c85-b5d6-47c7f2f6e1fd None None] MFA Rules not processed for user `6256407d9df648d5bbe33495cb9a8b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:24.177576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a8fa0354-7818-4c85-b5d6-47c7f2f6e1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:24.233545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1124/2254] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Sep 22 15:00:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:24.246076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ab9c4-44c8-4f3e-b645-f41848dc94c9 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:24.247513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-958ab9c4-44c8-4f3e-b645-f41848dc94c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:24.318782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ab9c4-44c8-4f3e-b645-f41848dc94c9 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:24.324340 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b95ab-8be2-4f8a-ba96-f04caebeb5a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a265623ae734936a340c97df852e227', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:24.324931 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b95ab-8be2-4f8a-ba96-f04caebeb5a8 service swift] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:24.325213 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b95ab-8be2-4f8a-ba96-f04caebeb5a8 service swift] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:24.325486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b95ab-8be2-4f8a-ba96-f04caebeb5a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:24.334990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b95ab-8be2-4f8a-ba96-f04caebeb5a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:24.337270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b95ab-8be2-4f8a-ba96-f04caebeb5a8 service swift] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:24.345771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1131/2255] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Sep 22 15:00:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:00:28.500506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39299b-335c-4aea-af94-d095c1850859 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:28.501428 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39299b-335c-4aea-af94-d095c1850859 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:28.503704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39299b-335c-4aea-af94-d095c1850859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:28.532134 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-5e39299b-335c-4aea-af94-d095c1850859 None None] MFA Rules not processed for user `f298f0c6a9e44efba2a1e2fc648a0f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:28.533877 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-5e39299b-335c-4aea-af94-d095c1850859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:28.614519 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1125/2256] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:00:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:34.345142 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=32 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:37.495855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c21f76-cfa3-4f22-8953-f347ca42cd1e None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:37.497018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c21f76-cfa3-4f22-8953-f347ca42cd1e None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:37.497018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c21f76-cfa3-4f22-8953-f347ca42cd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:37.528320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-64c21f76-cfa3-4f22-8953-f347ca42cd1e None None] MFA Rules not processed for user `038365d6ba7041f4af386767d98ed709`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:37.530014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-64c21f76-cfa3-4f22-8953-f347ca42cd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:37.603049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1132/2257] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:00:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:38.606870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=50 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:47.595001 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=25 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:47.864760 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd18907-ecc0-4d00-82e0-9bebd38ac976 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:47.865485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd18907-ecc0-4d00-82e0-9bebd38ac976 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:47.865485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd18907-ecc0-4d00-82e0-9bebd38ac976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:47.885856 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-6fd18907-ecc0-4d00-82e0-9bebd38ac976 None None] MFA Rules not processed for user `6256407d9df648d5bbe33495cb9a8b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:47.887126 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-6fd18907-ecc0-4d00-82e0-9bebd38ac976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:47.894867 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1126/2258] 10.222.0.26 () {58 vars in 967 bytes} [Thu Sep 22 15:00:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:47.909086 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c21f76-cfa3-4f22-8953-f347ca42cd1e None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:47.921313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c21f76-cfa3-4f22-8953-f347ca42cd1e None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:47.926508 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63895e65-2bf4-4afc-9001-9be06261939a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a265623ae734936a340c97df852e227', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:47.927405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63895e65-2bf4-4afc-9001-9be06261939a service swift] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:47.927693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63895e65-2bf4-4afc-9001-9be06261939a service swift] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:47.927967 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63895e65-2bf4-4afc-9001-9be06261939a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:47.937201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63895e65-2bf4-4afc-9001-9be06261939a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:47.939127 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63895e65-2bf4-4afc-9001-9be06261939a service swift] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:47.948571 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1133/2259] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Sep 22 15:00:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:00:48.392749 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f3f40-2102-4b8d-b119-993c80ffa4c3 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:48.393085 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f3f40-2102-4b8d-b119-993c80ffa4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:48.393306 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f3f40-2102-4b8d-b119-993c80ffa4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:48.414447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-446f3f40-2102-4b8d-b119-993c80ffa4c3 None None] MFA Rules not processed for user `6256407d9df648d5bbe33495cb9a8b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:48.415730 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-446f3f40-2102-4b8d-b119-993c80ffa4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:48.423201 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1127/2260] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Sep 22 15:00:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:48.437323 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63895e65-2bf4-4afc-9001-9be06261939a service swift] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:48.448538 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63895e65-2bf4-4afc-9001-9be06261939a service swift] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:48.453876 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ffcab-04d9-41d9-a87c-e6f7fa92aeb5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a265623ae734936a340c97df852e227', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:48.454485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ffcab-04d9-41d9-a87c-e6f7fa92aeb5 service swift] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:48.454848 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ffcab-04d9-41d9-a87c-e6f7fa92aeb5 service swift] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:48.455130 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ffcab-04d9-41d9-a87c-e6f7fa92aeb5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:48.464307 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ffcab-04d9-41d9-a87c-e6f7fa92aeb5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:48.466262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ffcab-04d9-41d9-a87c-e6f7fa92aeb5 service swift] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:48.477029 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1134/2261] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Sep 22 15:00:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:00:50.441020 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f3f40-2102-4b8d-b119-993c80ffa4c3 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:50.443744 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-446f3f40-2102-4b8d-b119-993c80ffa4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:50.469783 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f3f40-2102-4b8d-b119-993c80ffa4c3 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:50.480207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce65d66-16eb-463d-810b-f0a94ac3f666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:50.481081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce65d66-16eb-463d-810b-f0a94ac3f666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:50.481533 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce65d66-16eb-463d-810b-f0a94ac3f666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:50.481963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce65d66-16eb-463d-810b-f0a94ac3f666 admin admin] PATH_INFO: `/v3/users/6465d1250d2146b2be8d17a70b8bb929` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:50.488695 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce65d66-16eb-463d-810b-f0a94ac3f666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6465d1250d2146b2be8d17a70b8bb929)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:50.491679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce65d66-16eb-463d-810b-f0a94ac3f666 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:50.576359 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bce65d66-16eb-463d-810b-f0a94ac3f666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6465d1250d2146b2be8d17a70b8bb929'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:50.589985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bce65d66-16eb-463d-810b-f0a94ac3f666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6465d1250d2146b2be8d17a70b8bb929'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:50.590822 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bce65d66-16eb-463d-810b-f0a94ac3f666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6465d1250d2146b2be8d17a70b8bb929'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:50.617261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bce65d66-16eb-463d-810b-f0a94ac3f666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6465d1250d2146b2be8d17a70b8bb929'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:50.619439 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bce65d66-16eb-463d-810b-f0a94ac3f666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6465d1250d2146b2be8d17a70b8bb929'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:50.650338 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1128/2262] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:00:50 2022] DELETE /identity/v3/users/6465d1250d2146b2be8d17a70b8bb929 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:50.970676 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ffcab-04d9-41d9-a87c-e6f7fa92aeb5 service swift] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:50.988909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ffcab-04d9-41d9-a87c-e6f7fa92aeb5 service swift] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:51.092812 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0e11f-c8c3-46ed-afbb-8e629a6a44ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:51.094383 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0e11f-c8c3-46ed-afbb-8e629a6a44ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:51.094924 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0e11f-c8c3-46ed-afbb-8e629a6a44ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:51.095557 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0e11f-c8c3-46ed-afbb-8e629a6a44ac admin admin] PATH_INFO: `/v3/projects/495d28624eac48069fdff279e2f1b001` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:51.097515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0e11f-c8c3-46ed-afbb-8e629a6a44ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=495d28624eac48069fdff279e2f1b001)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:51.100315 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0e11f-c8c3-46ed-afbb-8e629a6a44ac admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:51.219668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-65c0e11f-c8c3-46ed-afbb-8e629a6a44ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '495d28624eac48069fdff279e2f1b001', 'request_id': 'req-65c0e11f-c8c3-46ed-afbb-8e629a6a44ac'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:51.235438 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-65c0e11f-c8c3-46ed-afbb-8e629a6a44ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '495d28624eac48069fdff279e2f1b001', 'request_id': 'req-65c0e11f-c8c3-46ed-afbb-8e629a6a44ac'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:51.247885 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-65c0e11f-c8c3-46ed-afbb-8e629a6a44ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '495d28624eac48069fdff279e2f1b001', 'request_id': 'req-65c0e11f-c8c3-46ed-afbb-8e629a6a44ac'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:51.287717 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-65c0e11f-c8c3-46ed-afbb-8e629a6a44ac admin admin] The token cache is being invalidate because project 495d28624eac48069fdff279e2f1b001 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:00:51.289363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-65c0e11f-c8c3-46ed-afbb-8e629a6a44ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:51.290047 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-65c0e11f-c8c3-46ed-afbb-8e629a6a44ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:00:51.292638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1135/2263] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:00:50 2022] DELETE /identity/v3/projects/495d28624eac48069fdff279e2f1b001 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:51.297164 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aca6d-6ed4-4bdd-a3d7-8519e6bf002e None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:51.297550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aca6d-6ed4-4bdd-a3d7-8519e6bf002e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:51.297923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aca6d-6ed4-4bdd-a3d7-8519e6bf002e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:51.332440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-d26aca6d-6ed4-4bdd-a3d7-8519e6bf002e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:51.336468 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d26aca6d-6ed4-4bdd-a3d7-8519e6bf002e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:51.486941 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1129/2264] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:51.499036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2baf6-899a-4617-80e0-827f89f998d4 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:51.499376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2baf6-899a-4617-80e0-827f89f998d4 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:51.499765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2baf6-899a-4617-80e0-827f89f998d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:51.579513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-b1c2baf6-899a-4617-80e0-827f89f998d4 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:51.581872 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b1c2baf6-899a-4617-80e0-827f89f998d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:51.596173 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1136/2265] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:51.596894 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26aca6d-6ed4-4bdd-a3d7-8519e6bf002e None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:51.608306 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26aca6d-6ed4-4bdd-a3d7-8519e6bf002e None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:51.621546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c03393-467e-4110-9e44-a9a0b5a68c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:51.622440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c03393-467e-4110-9e44-a9a0b5a68c62 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:51.626474 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c03393-467e-4110-9e44-a9a0b5a68c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:51.626639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c03393-467e-4110-9e44-a9a0b5a68c62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:51.627389 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c03393-467e-4110-9e44-a9a0b5a68c62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:00:51.627871 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c03393-467e-4110-9e44-a9a0b5a68c62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:51.629660 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c03393-467e-4110-9e44-a9a0b5a68c62 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:51.659204 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1130/2266] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 15:00:51.660321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2baf6-899a-4617-80e0-827f89f998d4 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:51.698903 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2baf6-899a-4617-80e0-827f89f998d4 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:51.707507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29e6d7-5740-4bd6-a4dd-96889137daa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:51.708185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29e6d7-5740-4bd6-a4dd-96889137daa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:51.709109 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29e6d7-5740-4bd6-a4dd-96889137daa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:51.717742 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29e6d7-5740-4bd6-a4dd-96889137daa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:51.719215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29e6d7-5740-4bd6-a4dd-96889137daa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:51.723425 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29e6d7-5740-4bd6-a4dd-96889137daa7 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:51.784742 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1137/2267] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:00:51 2022] POST /identity/v3/projects => generated 369 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:00:51.790103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c03393-467e-4110-9e44-a9a0b5a68c62 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:51.803532 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c03393-467e-4110-9e44-a9a0b5a68c62 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:51.898948 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:51.944868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3eaa4-e414-4219-832e-f319b3ad52ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:51.944868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3eaa4-e414-4219-832e-f319b3ad52ad admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:51.944868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3eaa4-e414-4219-832e-f319b3ad52ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:51.945305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3eaa4-e414-4219-832e-f319b3ad52ad admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:51.946919 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3eaa4-e414-4219-832e-f319b3ad52ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:51.952368 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3eaa4-e414-4219-832e-f319b3ad52ad admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:51.992876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1131/2268] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:00:51 2022] POST /identity/v3/users => generated 285 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:00:51.997611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29e6d7-5740-4bd6-a4dd-96889137daa7 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:52.014371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29e6d7-5740-4bd6-a4dd-96889137daa7 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:52.019493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72724267-ec0f-4c3b-aaf1-fa5b249d7b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:52.020104 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72724267-ec0f-4c3b-aaf1-fa5b249d7b6b admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:52.020377 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72724267-ec0f-4c3b-aaf1-fa5b249d7b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:52.020674 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72724267-ec0f-4c3b-aaf1-fa5b249d7b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:52.021724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72724267-ec0f-4c3b-aaf1-fa5b249d7b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:52.023667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72724267-ec0f-4c3b-aaf1-fa5b249d7b6b admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:52.040564 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1138/2269] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:00:52.041838 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3eaa4-e414-4219-832e-f319b3ad52ad admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:52.063416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3eaa4-e414-4219-832e-f319b3ad52ad admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:52.079401 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8528618-c450-4144-a18f-22b8732245e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:52.080444 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8528618-c450-4144-a18f-22b8732245e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:52.080799 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8528618-c450-4144-a18f-22b8732245e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:52.083186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8528618-c450-4144-a18f-22b8732245e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:52.087232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8528618-c450-4144-a18f-22b8732245e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:52.091284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8528618-c450-4144-a18f-22b8732245e8 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:52.112459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1132/2270] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:00:52.114500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72724267-ec0f-4c3b-aaf1-fa5b249d7b6b admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:52.125659 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72724267-ec0f-4c3b-aaf1-fa5b249d7b6b admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:52.144329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4502142-ac2f-4574-9a54-f523dd5c1297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:52.146183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4502142-ac2f-4574-9a54-f523dd5c1297 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:52.148468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4502142-ac2f-4574-9a54-f523dd5c1297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:52.150604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4502142-ac2f-4574-9a54-f523dd5c1297 admin admin] PATH_INFO: `/v3/projects/efe612769d354e6b8364c52ef9d15645/users/866b3dd9bbff47fc80b52876f72622e7/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:52.175572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4502142-ac2f-4574-9a54-f523dd5c1297 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efe612769d354e6b8364c52ef9d15645, user_id=866b3dd9bbff47fc80b52876f72622e7, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:52.183249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4502142-ac2f-4574-9a54-f523dd5c1297 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:52.196124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:00:52.204589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1139/2271] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:00:52 2022] PUT /identity/v3/projects/efe612769d354e6b8364c52ef9d15645/users/866b3dd9bbff47fc80b52876f72622e7/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:00:52.215082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc5753-1d5e-47c7-bc57-e8c4156c3851 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:52.215853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc5753-1d5e-47c7-bc57-e8c4156c3851 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:52.216323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc5753-1d5e-47c7-bc57-e8c4156c3851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:52.254419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-d5cc5753-1d5e-47c7-bc57-e8c4156c3851 None None] MFA Rules not processed for user `866b3dd9bbff47fc80b52876f72622e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:00:52.256007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d5cc5753-1d5e-47c7-bc57-e8c4156c3851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:52.377506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1133/2272] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:00:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:00:52.415368 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4502142-ac2f-4574-9a54-f523dd5c1297 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:00:52.417344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c4502142-ac2f-4574-9a54-f523dd5c1297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:00:52.498764 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4502142-ac2f-4574-9a54-f523dd5c1297 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:00:52.503887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b091a-f020-46b1-9589-550f5b4431e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5248cb1c39412680cce0d9891fe1ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:00:52.504707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b091a-f020-46b1-9589-550f5b4431e1 service glance] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:00:52.505443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b091a-f020-46b1-9589-550f5b4431e1 service glance] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:00:52.506124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b091a-f020-46b1-9589-550f5b4431e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:00:52.517619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b091a-f020-46b1-9589-550f5b4431e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:00:52.519574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b091a-f020-46b1-9589-550f5b4431e1 service glance] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:00:52.535102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1140/2273] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:00:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:01:02.371900 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:02.526567 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=26 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:08.892293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4671af-a7f2-4cff-bff5-2f9d654569ee None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:08.892796 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4671af-a7f2-4cff-bff5-2f9d654569ee None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:08.892796 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4671af-a7f2-4cff-bff5-2f9d654569ee None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:08.895299 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1134/2274] 10.222.0.26 () {54 vars in 921 bytes} [Thu Sep 22 15:01:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 15:01:08.902080 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d6282-e149-407f-98e9-61240d4d6e03 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:08.902565 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d6282-e149-407f-98e9-61240d4d6e03 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:08.902862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d6282-e149-407f-98e9-61240d4d6e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:08.923176 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-9f7d6282-e149-407f-98e9-61240d4d6e03 None None] MFA Rules not processed for user `0c9be41c39494e54a14398a1f03c4d69`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:08.924404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-9f7d6282-e149-407f-98e9-61240d4d6e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:08.996996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1141/2275] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Sep 22 15:01:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:09.011235 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4671af-a7f2-4cff-bff5-2f9d654569ee None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:09.013027 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ba4671af-a7f2-4cff-bff5-2f9d654569ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:09.027537 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4671af-a7f2-4cff-bff5-2f9d654569ee None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:09.031500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea268f-59dc-4ab9-99fb-4aba24e84c2b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5248cb1c39412680cce0d9891fe1ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:09.032383 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea268f-59dc-4ab9-99fb-4aba24e84c2b service glance] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:09.036063 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea268f-59dc-4ab9-99fb-4aba24e84c2b service glance] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:09.036364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea268f-59dc-4ab9-99fb-4aba24e84c2b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:09.047657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea268f-59dc-4ab9-99fb-4aba24e84c2b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:09.049133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea268f-59dc-4ab9-99fb-4aba24e84c2b service glance] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:09.061152 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1135/2276] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:01:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:01:09.130168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1f63e-1e38-477f-87e4-f966f1495bbe None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:09.130452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1f63e-1e38-477f-87e4-f966f1495bbe None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:09.130787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1f63e-1e38-477f-87e4-f966f1495bbe None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:09.132126 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1142/2277] 10.222.0.26 () {54 vars in 921 bytes} [Thu Sep 22 15:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 15:01:09.137579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c551f2-a0d1-4b46-a271-4e4c572c2729 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:09.137867 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c551f2-a0d1-4b46-a271-4e4c572c2729 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:09.138173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c551f2-a0d1-4b46-a271-4e4c572c2729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:09.158461 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-28c551f2-a0d1-4b46-a271-4e4c572c2729 None None] MFA Rules not processed for user `f77bbd8de87042aab3674db366635cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:09.159603 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-28c551f2-a0d1-4b46-a271-4e4c572c2729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:09.214753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1136/2278] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Sep 22 15:01:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:09.226502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1f63e-1e38-477f-87e4-f966f1495bbe None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:09.228288 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-30d1f63e-1e38-477f-87e4-f966f1495bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:09.241019 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1f63e-1e38-477f-87e4-f966f1495bbe None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:09.245062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5289c11d-2a91-4bc2-a9e3-6afc3171c291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:09.245642 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289c11d-2a91-4bc2-a9e3-6afc3171c291 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:09.245914 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289c11d-2a91-4bc2-a9e3-6afc3171c291 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:09.246180 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289c11d-2a91-4bc2-a9e3-6afc3171c291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:09.253062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289c11d-2a91-4bc2-a9e3-6afc3171c291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:09.254456 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5289c11d-2a91-4bc2-a9e3-6afc3171c291 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:09.262681 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1143/2279] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:01:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:01:09.376360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c2d3c-0e4a-47c5-83e9-9c88b88d6b0c None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:09.376643 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c2d3c-0e4a-47c5-83e9-9c88b88d6b0c None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:09.376918 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c2d3c-0e4a-47c5-83e9-9c88b88d6b0c None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:09.378300 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1137/2280] 10.222.0.26 () {54 vars in 921 bytes} [Thu Sep 22 15:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 15:01:09.383965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ffc06-c0c5-4b5d-a71e-1fefb0946929 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:09.384809 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ffc06-c0c5-4b5d-a71e-1fefb0946929 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:09.385087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ffc06-c0c5-4b5d-a71e-1fefb0946929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:09.404523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-f81ffc06-c0c5-4b5d-a71e-1fefb0946929 None None] MFA Rules not processed for user `0c9be41c39494e54a14398a1f03c4d69`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:09.405725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f81ffc06-c0c5-4b5d-a71e-1fefb0946929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:09.415461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1144/2281] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Sep 22 15:01:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:01:09.425836 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c2d3c-0e4a-47c5-83e9-9c88b88d6b0c None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:09.437434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c2d3c-0e4a-47c5-83e9-9c88b88d6b0c None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:09.440965 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea3d23-8b1e-4bdd-92b5-2073bb762708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:09.441563 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea3d23-8b1e-4bdd-92b5-2073bb762708 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:09.441838 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea3d23-8b1e-4bdd-92b5-2073bb762708 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:09.442107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea3d23-8b1e-4bdd-92b5-2073bb762708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:09.452681 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea3d23-8b1e-4bdd-92b5-2073bb762708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:09.454106 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea3d23-8b1e-4bdd-92b5-2073bb762708 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:09.465657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1138/2282] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:01:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:01:12.369024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f09c2-e015-4798-ab9e-54c65b2d516c None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:12.369502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f09c2-e015-4798-ab9e-54c65b2d516c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:12.369611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f09c2-e015-4798-ab9e-54c65b2d516c None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:12.370934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1145/2283] 10.222.0.26 () {54 vars in 961 bytes} [Thu Sep 22 15:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 15:01:12.376086 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df983029-bc3a-45f0-acdd-89ad76cbcb2f None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:12.376551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df983029-bc3a-45f0-acdd-89ad76cbcb2f None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:12.376730 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df983029-bc3a-45f0-acdd-89ad76cbcb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:12.395788 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-df983029-bc3a-45f0-acdd-89ad76cbcb2f None None] MFA Rules not processed for user `47b6b34b4ee5492fb1f0699716e5118b`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:12.397006 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-df983029-bc3a-45f0-acdd-89ad76cbcb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:12.459885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1139/2284] 10.222.0.26 () {58 vars in 1058 bytes} [Thu Sep 22 15:01:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:12.468764 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f09c2-e015-4798-ab9e-54c65b2d516c None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:12.482973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f09c2-e015-4798-ab9e-54c65b2d516c None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:12.488563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922ebb6-649b-4c1f-b6f7-2b62650528c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b6b34b4ee5492fb1f0699716e5118b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:12.489084 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922ebb6-649b-4c1f-b6f7-2b62650528c2 service placement] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:12.489365 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922ebb6-649b-4c1f-b6f7-2b62650528c2 service placement] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:12.489639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922ebb6-649b-4c1f-b6f7-2b62650528c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:12.491393 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1922ebb6-649b-4c1f-b6f7-2b62650528c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:12.503225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922ebb6-649b-4c1f-b6f7-2b62650528c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:12.504829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922ebb6-649b-4c1f-b6f7-2b62650528c2 service placement] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:12.515286 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1146/2285] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:01:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:01:18.695009 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1973684-85b4-4520-bc38-6940a2a08b4c None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:18.695603 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1973684-85b4-4520-bc38-6940a2a08b4c None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:18.695603 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1973684-85b4-4520-bc38-6940a2a08b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:18.716513 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-b1973684-85b4-4520-bc38-6940a2a08b4c None None] MFA Rules not processed for user `6256407d9df648d5bbe33495cb9a8b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:18.717898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b1973684-85b4-4520-bc38-6940a2a08b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:18.780686 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1140/2286] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Sep 22 15:01:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:18.805057 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922ebb6-649b-4c1f-b6f7-2b62650528c2 service placement] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:18.806567 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1922ebb6-649b-4c1f-b6f7-2b62650528c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:18.878743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922ebb6-649b-4c1f-b6f7-2b62650528c2 service placement] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:18.884639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733727a8-8811-4682-b021-0be02c86d0ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a265623ae734936a340c97df852e227', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:18.885274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733727a8-8811-4682-b021-0be02c86d0ad service swift] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:18.885553 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733727a8-8811-4682-b021-0be02c86d0ad service swift] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:18.885825 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733727a8-8811-4682-b021-0be02c86d0ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:18.894266 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733727a8-8811-4682-b021-0be02c86d0ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:18.895888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733727a8-8811-4682-b021-0be02c86d0ad service swift] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:18.905644 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1147/2287] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Sep 22 15:01:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 22 15:01:28.772623 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:28.901727 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=78 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:30.390953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1973684-85b4-4520-bc38-6940a2a08b4c None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:30.454119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1973684-85b4-4520-bc38-6940a2a08b4c None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:30.458655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efb7af-0b31-457d-985b-527cc9a28b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:30.459306 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efb7af-0b31-457d-985b-527cc9a28b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:30.459586 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efb7af-0b31-457d-985b-527cc9a28b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:30.459860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efb7af-0b31-457d-985b-527cc9a28b4c admin admin] PATH_INFO: `/v3/users/866b3dd9bbff47fc80b52876f72622e7` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:30.461030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efb7af-0b31-457d-985b-527cc9a28b4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=866b3dd9bbff47fc80b52876f72622e7)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:30.463257 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efb7af-0b31-457d-985b-527cc9a28b4c admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:30.508949 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e4efb7af-0b31-457d-985b-527cc9a28b4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '866b3dd9bbff47fc80b52876f72622e7'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:30.517103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e4efb7af-0b31-457d-985b-527cc9a28b4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '866b3dd9bbff47fc80b52876f72622e7'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:30.517709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e4efb7af-0b31-457d-985b-527cc9a28b4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '866b3dd9bbff47fc80b52876f72622e7'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:30.529699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e4efb7af-0b31-457d-985b-527cc9a28b4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '866b3dd9bbff47fc80b52876f72622e7'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:30.530319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e4efb7af-0b31-457d-985b-527cc9a28b4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '866b3dd9bbff47fc80b52876f72622e7'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:30.542283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1141/2288] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:01:30 2022] DELETE /identity/v3/users/866b3dd9bbff47fc80b52876f72622e7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:01:30.546794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37e01a-1ec7-4810-a3e9-38e79fe21f2a None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:30.547317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37e01a-1ec7-4810-a3e9-38e79fe21f2a None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:30.547428 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37e01a-1ec7-4810-a3e9-38e79fe21f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:30.567040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-cb37e01a-1ec7-4810-a3e9-38e79fe21f2a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:30.568235 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-cb37e01a-1ec7-4810-a3e9-38e79fe21f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:30.629353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1148/2289] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:01:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:30.639256 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efb7af-0b31-457d-985b-527cc9a28b4c admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:30.640791 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e4efb7af-0b31-457d-985b-527cc9a28b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:30.704232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efb7af-0b31-457d-985b-527cc9a28b4c admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:30.708074 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c8738-449b-439a-aae9-422514569e56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:30.708688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c8738-449b-439a-aae9-422514569e56 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:30.708966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c8738-449b-439a-aae9-422514569e56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:30.709237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c8738-449b-439a-aae9-422514569e56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:30.719427 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c8738-449b-439a-aae9-422514569e56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:30.720902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c8738-449b-439a-aae9-422514569e56 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:30.731456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1142/2290] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:01:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:01:31.060733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37e01a-1ec7-4810-a3e9-38e79fe21f2a None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:31.062162 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-cb37e01a-1ec7-4810-a3e9-38e79fe21f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:31.074034 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37e01a-1ec7-4810-a3e9-38e79fe21f2a None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:31.078615 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bc64d-6471-426f-941d-a5bcbfde19ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:31.079289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bc64d-6471-426f-941d-a5bcbfde19ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:31.079566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bc64d-6471-426f-941d-a5bcbfde19ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:31.079833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bc64d-6471-426f-941d-a5bcbfde19ec admin admin] PATH_INFO: `/v3/projects/efe612769d354e6b8364c52ef9d15645` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:31.080941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bc64d-6471-426f-941d-a5bcbfde19ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=efe612769d354e6b8364c52ef9d15645)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:31.082871 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bc64d-6471-426f-941d-a5bcbfde19ec admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:31.133567 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-632bc64d-6471-426f-941d-a5bcbfde19ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efe612769d354e6b8364c52ef9d15645', 'request_id': 'req-632bc64d-6471-426f-941d-a5bcbfde19ec'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:31.140158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-632bc64d-6471-426f-941d-a5bcbfde19ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efe612769d354e6b8364c52ef9d15645', 'request_id': 'req-632bc64d-6471-426f-941d-a5bcbfde19ec'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:31.146338 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-632bc64d-6471-426f-941d-a5bcbfde19ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efe612769d354e6b8364c52ef9d15645', 'request_id': 'req-632bc64d-6471-426f-941d-a5bcbfde19ec'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:31.159488 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-632bc64d-6471-426f-941d-a5bcbfde19ec admin admin] The token cache is being invalidate because project efe612769d354e6b8364c52ef9d15645 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:01:31.159780 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-632bc64d-6471-426f-941d-a5bcbfde19ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:31.160407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-632bc64d-6471-426f-941d-a5bcbfde19ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:01:31.161903 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1149/2291] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:01:31 2022] DELETE /identity/v3/projects/efe612769d354e6b8364c52ef9d15645 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:01:40.726693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:41.156319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:01:45.991600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318ea56-96f1-42bb-8e44-e9eef51ca9dc None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:45.992097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318ea56-96f1-42bb-8e44-e9eef51ca9dc None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:45.992201 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318ea56-96f1-42bb-8e44-e9eef51ca9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:46.012032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-e318ea56-96f1-42bb-8e44-e9eef51ca9dc None None] MFA Rules not processed for user `6256407d9df648d5bbe33495cb9a8b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:46.013222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e318ea56-96f1-42bb-8e44-e9eef51ca9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:46.067925 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1143/2292] 10.222.0.26 () {58 vars in 967 bytes} [Thu Sep 22 15:01:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:46.080696 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bc64d-6471-426f-941d-a5bcbfde19ec admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:01:46.082108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-632bc64d-6471-426f-941d-a5bcbfde19ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:46.153160 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bc64d-6471-426f-941d-a5bcbfde19ec admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:01:46.158460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6e9eb-639b-4468-9745-fcd6e4c01dcf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a265623ae734936a340c97df852e227', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:01:46.159110 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6e9eb-639b-4468-9745-fcd6e4c01dcf service swift] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:46.159390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6e9eb-639b-4468-9745-fcd6e4c01dcf service swift] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:46.159662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6e9eb-639b-4468-9745-fcd6e4c01dcf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:46.168512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6e9eb-639b-4468-9745-fcd6e4c01dcf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:01:46.170112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6e9eb-639b-4468-9745-fcd6e4c01dcf service swift] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:01:46.179407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1150/2293] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Sep 22 15:01:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:01:54.022096 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9007404-75ae-4d99-91eb-ec3c932e2cc4 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:01:54.022578 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9007404-75ae-4d99-91eb-ec3c932e2cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:01:54.022705 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9007404-75ae-4d99-91eb-ec3c932e2cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:01:54.043359 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-d9007404-75ae-4d99-91eb-ec3c932e2cc4 None None] MFA Rules not processed for user `8463811a630e4b90b1cd96e4dec020a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:01:54.044604 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d9007404-75ae-4d99-91eb-ec3c932e2cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:01:54.099669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1144/2294] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:01:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:01:56.175183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=32 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:02:04.093035 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=50 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:02:09.929280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b0597-7f2f-47df-a3f7-b3e1726c9720 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:09.929792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b0597-7f2f-47df-a3f7-b3e1726c9720 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:09.929884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b0597-7f2f-47df-a3f7-b3e1726c9720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:09.955413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-a77b0597-7f2f-47df-a3f7-b3e1726c9720 None None] MFA Rules not processed for user `f298f0c6a9e44efba2a1e2fc648a0f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:09.956637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a77b0597-7f2f-47df-a3f7-b3e1726c9720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:10.015675 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1151/2295] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:02:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:02:20.008294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=25 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:02:27.130302 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3432ed-bcdc-4a2c-b27d-d7d772dedb2e None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:27.131270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3432ed-bcdc-4a2c-b27d-d7d772dedb2e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:27.131534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3432ed-bcdc-4a2c-b27d-d7d772dedb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:27.150441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-ec3432ed-bcdc-4a2c-b27d-d7d772dedb2e None None] MFA Rules not processed for user `038365d6ba7041f4af386767d98ed709`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:27.152259 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ec3432ed-bcdc-4a2c-b27d-d7d772dedb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:27.209189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1145/2296] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:02:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:02:37.202457 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=25 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:02:49.327987 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703385e-0f90-4140-a22b-a3cb4fdb71ad None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:49.328609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703385e-0f90-4140-a22b-a3cb4fdb71ad None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:49.328609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703385e-0f90-4140-a22b-a3cb4fdb71ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:49.349001 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-d703385e-0f90-4140-a22b-a3cb4fdb71ad None None] MFA Rules not processed for user `23d683cdc90041b5b29767fa2e2107e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:02:49.350225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d703385e-0f90-4140-a22b-a3cb4fdb71ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:49.407741 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1152/2297] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:02:56.989129 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3432ed-bcdc-4a2c-b27d-d7d772dedb2e None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:56.990567 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ec3432ed-bcdc-4a2c-b27d-d7d772dedb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:02:57.064831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3432ed-bcdc-4a2c-b27d-d7d772dedb2e None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:57.070926 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441abf8-8564-4079-a5e9-5f462c567489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:57.072262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441abf8-8564-4079-a5e9-5f462c567489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:57.072571 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441abf8-8564-4079-a5e9-5f462c567489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:57.072850 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441abf8-8564-4079-a5e9-5f462c567489 admin admin] PATH_INFO: `/v3/users/8463811a630e4b90b1cd96e4dec020a2` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:57.074134 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441abf8-8564-4079-a5e9-5f462c567489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8463811a630e4b90b1cd96e4dec020a2)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:57.076103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441abf8-8564-4079-a5e9-5f462c567489 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:57.128050 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f441abf8-8564-4079-a5e9-5f462c567489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8463811a630e4b90b1cd96e4dec020a2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:57.141103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f441abf8-8564-4079-a5e9-5f462c567489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8463811a630e4b90b1cd96e4dec020a2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:57.141982 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f441abf8-8564-4079-a5e9-5f462c567489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8463811a630e4b90b1cd96e4dec020a2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:57.160922 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f441abf8-8564-4079-a5e9-5f462c567489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8463811a630e4b90b1cd96e4dec020a2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:57.161718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f441abf8-8564-4079-a5e9-5f462c567489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8463811a630e4b90b1cd96e4dec020a2'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:57.174643 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1146/2298] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:02:56 2022] DELETE /identity/v3/users/8463811a630e4b90b1cd96e4dec020a2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:02:57.342959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703385e-0f90-4140-a22b-a3cb4fdb71ad None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:02:57.354581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703385e-0f90-4140-a22b-a3cb4fdb71ad None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:02:57.413956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cfd968-6f77-43e7-95d4-7e8d38571e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:02:57.414630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfd968-6f77-43e7-95d4-7e8d38571e14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:02:57.414936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfd968-6f77-43e7-95d4-7e8d38571e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:02:57.415210 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfd968-6f77-43e7-95d4-7e8d38571e14 admin admin] PATH_INFO: `/v3/projects/26875c24ce944501a6b23188c784d9f5` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:02:57.416345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cfd968-6f77-43e7-95d4-7e8d38571e14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26875c24ce944501a6b23188c784d9f5)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:02:57.418843 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cfd968-6f77-43e7-95d4-7e8d38571e14 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:02:57.465533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c9cfd968-6f77-43e7-95d4-7e8d38571e14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26875c24ce944501a6b23188c784d9f5', 'request_id': 'req-c9cfd968-6f77-43e7-95d4-7e8d38571e14'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:57.471648 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c9cfd968-6f77-43e7-95d4-7e8d38571e14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26875c24ce944501a6b23188c784d9f5', 'request_id': 'req-c9cfd968-6f77-43e7-95d4-7e8d38571e14'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:57.477152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c9cfd968-6f77-43e7-95d4-7e8d38571e14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26875c24ce944501a6b23188c784d9f5', 'request_id': 'req-c9cfd968-6f77-43e7-95d4-7e8d38571e14'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:57.489859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c9cfd968-6f77-43e7-95d4-7e8d38571e14 admin admin] The token cache is being invalidate because project 26875c24ce944501a6b23188c784d9f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:02:57.490152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c9cfd968-6f77-43e7-95d4-7e8d38571e14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:57.490811 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c9cfd968-6f77-43e7-95d4-7e8d38571e14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:02:57.492320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1153/2299] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:02:57 2022] DELETE /identity/v3/projects/26875c24ce944501a6b23188c784d9f5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:03:07.171058 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:07.486688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:09.586531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441abf8-8564-4079-a5e9-5f462c567489 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:09.588026 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-f441abf8-8564-4079-a5e9-5f462c567489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:09.645531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441abf8-8564-4079-a5e9-5f462c567489 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:09.693154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6fc0e-1b0d-4611-b771-793a4c08cf09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:09.693766 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6fc0e-1b0d-4611-b771-793a4c08cf09 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:09.694045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6fc0e-1b0d-4611-b771-793a4c08cf09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:09.694317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6fc0e-1b0d-4611-b771-793a4c08cf09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:09.696076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-05d6fc0e-1b0d-4611-b771-793a4c08cf09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:09.766541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6fc0e-1b0d-4611-b771-793a4c08cf09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:09.768013 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6fc0e-1b0d-4611-b771-793a4c08cf09 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:09.779723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1147/2300] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:12.356934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cfd968-6f77-43e7-95d4-7e8d38571e14 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:12.358348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c9cfd968-6f77-43e7-95d4-7e8d38571e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:12.418229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cfd968-6f77-43e7-95d4-7e8d38571e14 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:12.422004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01426e-7315-4d4f-a08a-c6754dd53e8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:12.422604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01426e-7315-4d4f-a08a-c6754dd53e8e service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:12.422906 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01426e-7315-4d4f-a08a-c6754dd53e8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:12.423176 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01426e-7315-4d4f-a08a-c6754dd53e8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:12.424887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-ea01426e-7315-4d4f-a08a-c6754dd53e8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:12.438146 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01426e-7315-4d4f-a08a-c6754dd53e8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:12.439607 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01426e-7315-4d4f-a08a-c6754dd53e8e service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:12.451782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1154/2301] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:16.737365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6fc0e-1b0d-4611-b771-793a4c08cf09 service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:16.738881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-05d6fc0e-1b0d-4611-b771-793a4c08cf09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:16.752635 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6fc0e-1b0d-4611-b771-793a4c08cf09 service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:16.758159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951976c0-598c-42b1-bffa-5396baf277bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:16.758788 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951976c0-598c-42b1-bffa-5396baf277bb service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:16.759128 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951976c0-598c-42b1-bffa-5396baf277bb service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:16.759369 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951976c0-598c-42b1-bffa-5396baf277bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:16.761088 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-951976c0-598c-42b1-bffa-5396baf277bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:16.817261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:16.820090 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951976c0-598c-42b1-bffa-5396baf277bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:16.821949 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951976c0-598c-42b1-bffa-5396baf277bb service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:16.831347 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1148/2302] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:16 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:17.906089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01426e-7315-4d4f-a08a-c6754dd53e8e service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:17.907572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-ea01426e-7315-4d4f-a08a-c6754dd53e8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:17.919443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01426e-7315-4d4f-a08a-c6754dd53e8e service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:17.923078 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da5aad-49b3-4a62-b682-6afbe122f5df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:17.923673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da5aad-49b3-4a62-b682-6afbe122f5df service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:17.923951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da5aad-49b3-4a62-b682-6afbe122f5df service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:17.924225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35da5aad-49b3-4a62-b682-6afbe122f5df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:17.925929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-35da5aad-49b3-4a62-b682-6afbe122f5df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:17.984270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35da5aad-49b3-4a62-b682-6afbe122f5df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:17.985860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35da5aad-49b3-4a62-b682-6afbe122f5df service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:17.994925 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1155/2303] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:26.827127 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:27.990644 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=54 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:32.833722 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345e834-828c-49db-9403-d5a027d48442 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:32.834263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345e834-828c-49db-9403-d5a027d48442 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:32.834263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345e834-828c-49db-9403-d5a027d48442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:32.853560 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-7345e834-828c-49db-9403-d5a027d48442 None None] MFA Rules not processed for user `f298f0c6a9e44efba2a1e2fc648a0f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:32.855425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7345e834-828c-49db-9403-d5a027d48442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:32.862556 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1149/2304] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:03:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:33.165842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da5aad-49b3-4a62-b682-6afbe122f5df service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:33.175904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35da5aad-49b3-4a62-b682-6afbe122f5df service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:33.179523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf75cd-5b63-41e8-a5d6-ccf4b51e73ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:33.180124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf75cd-5b63-41e8-a5d6-ccf4b51e73ab service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:33.180405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf75cd-5b63-41e8-a5d6-ccf4b51e73ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:33.180679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf75cd-5b63-41e8-a5d6-ccf4b51e73ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:33.182414 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3caf75cd-5b63-41e8-a5d6-ccf4b51e73ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:33.191926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf75cd-5b63-41e8-a5d6-ccf4b51e73ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:33.193345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf75cd-5b63-41e8-a5d6-ccf4b51e73ab service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:33.201191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1156/2305] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:33.513198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345e834-828c-49db-9403-d5a027d48442 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:33.514678 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7345e834-828c-49db-9403-d5a027d48442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:33.585591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345e834-828c-49db-9403-d5a027d48442 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:33.591544 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81011725-3da5-4aad-b42f-cd7666b64c82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b6b34b4ee5492fb1f0699716e5118b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:33.592171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81011725-3da5-4aad-b42f-cd7666b64c82 service placement] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:33.592427 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81011725-3da5-4aad-b42f-cd7666b64c82 service placement] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:33.592698 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81011725-3da5-4aad-b42f-cd7666b64c82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:33.594442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-81011725-3da5-4aad-b42f-cd7666b64c82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:33.606979 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81011725-3da5-4aad-b42f-cd7666b64c82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:33.608685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81011725-3da5-4aad-b42f-cd7666b64c82 service placement] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:33.618829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1150/2306] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:36.198898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf75cd-5b63-41e8-a5d6-ccf4b51e73ab service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:36.201014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3caf75cd-5b63-41e8-a5d6-ccf4b51e73ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:36.211688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf75cd-5b63-41e8-a5d6-ccf4b51e73ab service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:36.215905 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840c44a-6881-47f3-a0ea-6e329f8b1767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:36.216507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840c44a-6881-47f3-a0ea-6e329f8b1767 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:36.216779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840c44a-6881-47f3-a0ea-6e329f8b1767 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:36.217045 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840c44a-6881-47f3-a0ea-6e329f8b1767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:36.218772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-6840c44a-6881-47f3-a0ea-6e329f8b1767 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:36.288385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840c44a-6881-47f3-a0ea-6e329f8b1767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:36.289993 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840c44a-6881-47f3-a0ea-6e329f8b1767 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:36.301719 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1157/2307] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:36.883958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81011725-3da5-4aad-b42f-cd7666b64c82 service placement] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:36.897972 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81011725-3da5-4aad-b42f-cd7666b64c82 service placement] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:36.904730 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c349a7c-1296-43e8-a4ec-9fcf2cbd1262 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:36.905356 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c349a7c-1296-43e8-a4ec-9fcf2cbd1262 service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:36.906031 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c349a7c-1296-43e8-a4ec-9fcf2cbd1262 service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:36.906031 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c349a7c-1296-43e8-a4ec-9fcf2cbd1262 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:36.907685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-9c349a7c-1296-43e8-a4ec-9fcf2cbd1262 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:36.975222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c349a7c-1296-43e8-a4ec-9fcf2cbd1262 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:36.976747 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c349a7c-1296-43e8-a4ec-9fcf2cbd1262 service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:36.986351 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1151/2308] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:37.487300 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840c44a-6881-47f3-a0ea-6e329f8b1767 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:37.497136 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840c44a-6881-47f3-a0ea-6e329f8b1767 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:37.500846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079804e2-b7fb-48cd-83b7-67fbebab0482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:37.501452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079804e2-b7fb-48cd-83b7-67fbebab0482 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:37.501732 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079804e2-b7fb-48cd-83b7-67fbebab0482 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:37.502008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079804e2-b7fb-48cd-83b7-67fbebab0482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:37.503798 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-079804e2-b7fb-48cd-83b7-67fbebab0482 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:37.515073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079804e2-b7fb-48cd-83b7-67fbebab0482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:37.516511 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079804e2-b7fb-48cd-83b7-67fbebab0482 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:37.525278 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1158/2309] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:40.918920 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c349a7c-1296-43e8-a4ec-9fcf2cbd1262 service nova] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:40.920468 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-9c349a7c-1296-43e8-a4ec-9fcf2cbd1262 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:40.934812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c349a7c-1296-43e8-a4ec-9fcf2cbd1262 service nova] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:40.940225 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16ff61-3df0-4860-b58e-715865d00aa3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b6b34b4ee5492fb1f0699716e5118b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:40.940904 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16ff61-3df0-4860-b58e-715865d00aa3 service placement] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:40.941179 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16ff61-3df0-4860-b58e-715865d00aa3 service placement] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:40.941447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16ff61-3df0-4860-b58e-715865d00aa3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:40.943246 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-8f16ff61-3df0-4860-b58e-715865d00aa3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:40.956243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16ff61-3df0-4860-b58e-715865d00aa3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:40.957885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16ff61-3df0-4860-b58e-715865d00aa3 service placement] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:40.970093 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1152/2310] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:41.239627 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079804e2-b7fb-48cd-83b7-67fbebab0482 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:41.241097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-079804e2-b7fb-48cd-83b7-67fbebab0482 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:41.257243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079804e2-b7fb-48cd-83b7-67fbebab0482 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:41.266024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3eb11-483f-47e0-9ae3-fe7d0f44cd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:41.266824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3eb11-483f-47e0-9ae3-fe7d0f44cd36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:41.267050 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3eb11-483f-47e0-9ae3-fe7d0f44cd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:41.267284 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3eb11-483f-47e0-9ae3-fe7d0f44cd36 admin admin] PATH_INFO: `/v3/users/038365d6ba7041f4af386767d98ed709` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:41.269401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3eb11-483f-47e0-9ae3-fe7d0f44cd36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=038365d6ba7041f4af386767d98ed709)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:41.271525 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3eb11-483f-47e0-9ae3-fe7d0f44cd36 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:41.331191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1fe3eb11-483f-47e0-9ae3-fe7d0f44cd36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '038365d6ba7041f4af386767d98ed709'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:41.342711 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1fe3eb11-483f-47e0-9ae3-fe7d0f44cd36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '038365d6ba7041f4af386767d98ed709'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:41.351611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1fe3eb11-483f-47e0-9ae3-fe7d0f44cd36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '038365d6ba7041f4af386767d98ed709'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:41.365059 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1fe3eb11-483f-47e0-9ae3-fe7d0f44cd36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '038365d6ba7041f4af386767d98ed709'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:41.365762 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1fe3eb11-483f-47e0-9ae3-fe7d0f44cd36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '038365d6ba7041f4af386767d98ed709'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:41.372378 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1159/2311] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:03:41 2022] DELETE /identity/v3/users/038365d6ba7041f4af386767d98ed709 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:03:41.540901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16ff61-3df0-4860-b58e-715865d00aa3 service placement] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:41.552533 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16ff61-3df0-4860-b58e-715865d00aa3 service placement] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:41.616629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d636f39d-5640-4a51-8216-48c74f1c5ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:41.617254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636f39d-5640-4a51-8216-48c74f1c5ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:41.617531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636f39d-5640-4a51-8216-48c74f1c5ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:41.617813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d636f39d-5640-4a51-8216-48c74f1c5ea7 admin admin] PATH_INFO: `/v3/projects/6a15aa877b1b482fa70f65ecd4f8d1f3` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:41.618964 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636f39d-5640-4a51-8216-48c74f1c5ea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a15aa877b1b482fa70f65ecd4f8d1f3)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:41.620866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d636f39d-5640-4a51-8216-48c74f1c5ea7 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:41.633403 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:41.675680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d636f39d-5640-4a51-8216-48c74f1c5ea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a15aa877b1b482fa70f65ecd4f8d1f3', 'request_id': 'req-d636f39d-5640-4a51-8216-48c74f1c5ea7'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:41.683629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d636f39d-5640-4a51-8216-48c74f1c5ea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a15aa877b1b482fa70f65ecd4f8d1f3', 'request_id': 'req-d636f39d-5640-4a51-8216-48c74f1c5ea7'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:41.691459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d636f39d-5640-4a51-8216-48c74f1c5ea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a15aa877b1b482fa70f65ecd4f8d1f3', 'request_id': 'req-d636f39d-5640-4a51-8216-48c74f1c5ea7'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:41.702450 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d636f39d-5640-4a51-8216-48c74f1c5ea7 admin admin] The token cache is being invalidate because project 6a15aa877b1b482fa70f65ecd4f8d1f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:03:41.702764 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d636f39d-5640-4a51-8216-48c74f1c5ea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:41.703513 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d636f39d-5640-4a51-8216-48c74f1c5ea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:41.704929 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1153/2312] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:03:41 2022] DELETE /identity/v3/projects/6a15aa877b1b482fa70f65ecd4f8d1f3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:03:41.709955 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031244b-b168-4af3-936d-e26f9611e840 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:41.710235 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031244b-b168-4af3-936d-e26f9611e840 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:41.710508 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031244b-b168-4af3-936d-e26f9611e840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:41.730242 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-a031244b-b168-4af3-936d-e26f9611e840 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:41.731954 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a031244b-b168-4af3-936d-e26f9611e840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:41.766484 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:41.794158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1160/2313] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:41.801877 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7cd81-5e40-469f-afb0-c795b49494e5 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:41.802161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7cd81-5e40-469f-afb0-c795b49494e5 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:41.802434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7cd81-5e40-469f-afb0-c795b49494e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:41.822489 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-c2d7cd81-5e40-469f-afb0-c795b49494e5 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:41.823750 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-c2d7cd81-5e40-469f-afb0-c795b49494e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:41.831610 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1154/2314] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:41.836123 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031244b-b168-4af3-936d-e26f9611e840 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:41.845841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031244b-b168-4af3-936d-e26f9611e840 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:41.850429 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f2ea5-f053-4387-941c-444aaaf7dfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:41.851078 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f2ea5-f053-4387-941c-444aaaf7dfef admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:41.851360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f2ea5-f053-4387-941c-444aaaf7dfef admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:41.851632 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f2ea5-f053-4387-941c-444aaaf7dfef admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:41.852277 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f2ea5-f053-4387-941c-444aaaf7dfef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:03:41.852610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f2ea5-f053-4387-941c-444aaaf7dfef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:41.854229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f2ea5-f053-4387-941c-444aaaf7dfef admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:41.867999 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1161/2315] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:03:41.872940 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7cd81-5e40-469f-afb0-c795b49494e5 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:41.882178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7cd81-5e40-469f-afb0-c795b49494e5 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:41.886967 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc0582-65c6-4bee-b851-1a0a896602d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:41.887564 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc0582-65c6-4bee-b851-1a0a896602d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:41.887913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc0582-65c6-4bee-b851-1a0a896602d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:41.888188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afc0582-65c6-4bee-b851-1a0a896602d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:41.889089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc0582-65c6-4bee-b851-1a0a896602d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:41.892758 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afc0582-65c6-4bee-b851-1a0a896602d3 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:41.923505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1155/2316] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:03:41 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:03:41.927685 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f2ea5-f053-4387-941c-444aaaf7dfef admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:41.938569 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f2ea5-f053-4387-941c-444aaaf7dfef admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:41.997314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac087aa-0b6b-4216-997a-967326a27aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:41.997838 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac087aa-0b6b-4216-997a-967326a27aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:41.998118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac087aa-0b6b-4216-997a-967326a27aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:41.998391 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac087aa-0b6b-4216-997a-967326a27aa6 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:41.999065 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac087aa-0b6b-4216-997a-967326a27aa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:42.000996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac087aa-0b6b-4216-997a-967326a27aa6 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:42.024852 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1162/2317] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:03:41 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:03:42.026182 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc0582-65c6-4bee-b851-1a0a896602d3 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:42.037204 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afc0582-65c6-4bee-b851-1a0a896602d3 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:42.041934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9970e59-bdd3-48e9-a5fc-58c1e2817a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:42.042536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9970e59-bdd3-48e9-a5fc-58c1e2817a44 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:42.042856 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9970e59-bdd3-48e9-a5fc-58c1e2817a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:42.043153 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9970e59-bdd3-48e9-a5fc-58c1e2817a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:42.044367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9970e59-bdd3-48e9-a5fc-58c1e2817a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:42.045996 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9970e59-bdd3-48e9-a5fc-58c1e2817a44 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:42.056385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1156/2318] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:03:42.060532 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac087aa-0b6b-4216-997a-967326a27aa6 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:42.071292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac087aa-0b6b-4216-997a-967326a27aa6 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:42.076117 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ea292-3e23-48bf-a77c-4d4b7198339b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:42.076715 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ea292-3e23-48bf-a77c-4d4b7198339b admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:42.076993 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ea292-3e23-48bf-a77c-4d4b7198339b admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:42.077273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ea292-3e23-48bf-a77c-4d4b7198339b admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:42.077918 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ea292-3e23-48bf-a77c-4d4b7198339b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:42.079568 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ea292-3e23-48bf-a77c-4d4b7198339b admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:42.091047 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1163/2319] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:03:42.095390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9970e59-bdd3-48e9-a5fc-58c1e2817a44 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:42.105513 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9970e59-bdd3-48e9-a5fc-58c1e2817a44 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:42.111034 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7fba5-496f-4570-9834-06ecc273b2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:42.111685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7fba5-496f-4570-9834-06ecc273b2cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:42.111966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7fba5-496f-4570-9834-06ecc273b2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:42.112257 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7fba5-496f-4570-9834-06ecc273b2cf admin admin] PATH_INFO: `/v3/projects/95be18c5c5af452fb559080cff5528b3/users/b4ffc1e77a3047cc915afd88d4d9cd94/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:42.126708 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7fba5-496f-4570-9834-06ecc273b2cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=95be18c5c5af452fb559080cff5528b3, user_id=b4ffc1e77a3047cc915afd88d4d9cd94, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:42.129406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7fba5-496f-4570-9834-06ecc273b2cf admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:42.142140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1157/2320] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:03:42 2022] PUT /identity/v3/projects/95be18c5c5af452fb559080cff5528b3/users/b4ffc1e77a3047cc915afd88d4d9cd94/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:03:42.153057 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420f70e-1481-48f2-886e-5ca38faad856 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:42.153845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420f70e-1481-48f2-886e-5ca38faad856 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:42.154114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420f70e-1481-48f2-886e-5ca38faad856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:42.182412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-a420f70e-1481-48f2-886e-5ca38faad856 None None] MFA Rules not processed for user `b4ffc1e77a3047cc915afd88d4d9cd94`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:42.183662 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a420f70e-1481-48f2-886e-5ca38faad856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:42.253767 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1164/2321] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:03:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:42.266404 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7fba5-496f-4570-9834-06ecc273b2cf admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:42.268447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-6cb7fba5-496f-4570-9834-06ecc273b2cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:42.345661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7fba5-496f-4570-9834-06ecc273b2cf admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:42.351865 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee867e7-0743-49e7-93b3-7eaa7afbb17f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:42.352478 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee867e7-0743-49e7-93b3-7eaa7afbb17f service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:42.352759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee867e7-0743-49e7-93b3-7eaa7afbb17f service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:42.353028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee867e7-0743-49e7-93b3-7eaa7afbb17f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:42.363658 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee867e7-0743-49e7-93b3-7eaa7afbb17f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:42.365239 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee867e7-0743-49e7-93b3-7eaa7afbb17f service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:42.374987 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1158/2322] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:42.604136 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420f70e-1481-48f2-886e-5ca38faad856 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:42.646969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:42.673187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420f70e-1481-48f2-886e-5ca38faad856 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:42.677863 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b61e6-0320-44dc-8a9d-b96e4ca5e15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:42.678507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b61e6-0320-44dc-8a9d-b96e4ca5e15d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:42.680491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b61e6-0320-44dc-8a9d-b96e4ca5e15d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:42.680491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b61e6-0320-44dc-8a9d-b96e4ca5e15d admin admin] PATH_INFO: `/v3/users/b4ffc1e77a3047cc915afd88d4d9cd94` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:42.681446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b61e6-0320-44dc-8a9d-b96e4ca5e15d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4ffc1e77a3047cc915afd88d4d9cd94)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:42.683429 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8b61e6-0320-44dc-8a9d-b96e4ca5e15d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:42.730866 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-4f8b61e6-0320-44dc-8a9d-b96e4ca5e15d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4ffc1e77a3047cc915afd88d4d9cd94'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:42.738152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-4f8b61e6-0320-44dc-8a9d-b96e4ca5e15d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4ffc1e77a3047cc915afd88d4d9cd94'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:42.745857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-4f8b61e6-0320-44dc-8a9d-b96e4ca5e15d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4ffc1e77a3047cc915afd88d4d9cd94'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:42.759874 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-4f8b61e6-0320-44dc-8a9d-b96e4ca5e15d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4ffc1e77a3047cc915afd88d4d9cd94'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:42.760521 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-4f8b61e6-0320-44dc-8a9d-b96e4ca5e15d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4ffc1e77a3047cc915afd88d4d9cd94'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:42.767527 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1165/2323] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:03:42 2022] DELETE /identity/v3/users/b4ffc1e77a3047cc915afd88d4d9cd94 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:03:42.772102 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d11da-e283-4dfe-b1fd-6ab6303847d4 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:42.772391 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d11da-e283-4dfe-b1fd-6ab6303847d4 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:42.772666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d11da-e283-4dfe-b1fd-6ab6303847d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:42.793373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-cc2d11da-e283-4dfe-b1fd-6ab6303847d4 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:42.795013 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-cc2d11da-e283-4dfe-b1fd-6ab6303847d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:42.811130 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:42.868493 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1159/2324] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:42.879184 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b61e6-0320-44dc-8a9d-b96e4ca5e15d admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:42.880698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4f8b61e6-0320-44dc-8a9d-b96e4ca5e15d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:42.944725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b61e6-0320-44dc-8a9d-b96e4ca5e15d admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:42.948557 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f188a75-e123-42b1-b3e3-2e818360656a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:42.949196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f188a75-e123-42b1-b3e3-2e818360656a service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:42.949481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f188a75-e123-42b1-b3e3-2e818360656a service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:42.949727 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f188a75-e123-42b1-b3e3-2e818360656a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:42.960155 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f188a75-e123-42b1-b3e3-2e818360656a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:42.961844 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f188a75-e123-42b1-b3e3-2e818360656a service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:42.971862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1166/2325] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:43.247874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d11da-e283-4dfe-b1fd-6ab6303847d4 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:43.249547 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-cc2d11da-e283-4dfe-b1fd-6ab6303847d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:43.261454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d11da-e283-4dfe-b1fd-6ab6303847d4 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:43.266191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b091f8-d14e-449c-b686-f9de280dcc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:43.266884 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b091f8-d14e-449c-b686-f9de280dcc47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:43.267145 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b091f8-d14e-449c-b686-f9de280dcc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:43.267417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b091f8-d14e-449c-b686-f9de280dcc47 admin admin] PATH_INFO: `/v3/projects/95be18c5c5af452fb559080cff5528b3` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:43.268576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b091f8-d14e-449c-b686-f9de280dcc47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95be18c5c5af452fb559080cff5528b3)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:43.270484 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b091f8-d14e-449c-b686-f9de280dcc47 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:43.320294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b5b091f8-d14e-449c-b686-f9de280dcc47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95be18c5c5af452fb559080cff5528b3', 'request_id': 'req-b5b091f8-d14e-449c-b686-f9de280dcc47'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:43.327874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b5b091f8-d14e-449c-b686-f9de280dcc47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95be18c5c5af452fb559080cff5528b3', 'request_id': 'req-b5b091f8-d14e-449c-b686-f9de280dcc47'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:43.334378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b5b091f8-d14e-449c-b686-f9de280dcc47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95be18c5c5af452fb559080cff5528b3', 'request_id': 'req-b5b091f8-d14e-449c-b686-f9de280dcc47'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:43.345281 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b5b091f8-d14e-449c-b686-f9de280dcc47 admin admin] The token cache is being invalidate because project 95be18c5c5af452fb559080cff5528b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:03:43.345569 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b5b091f8-d14e-449c-b686-f9de280dcc47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:43.346212 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b5b091f8-d14e-449c-b686-f9de280dcc47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:03:43.347766 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1160/2326] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:03:43 2022] DELETE /identity/v3/projects/95be18c5c5af452fb559080cff5528b3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:03:43.352645 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b60f3-ff6a-4797-b976-31ae9844452c None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:43.353132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b60f3-ff6a-4797-b976-31ae9844452c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:43.353231 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b60f3-ff6a-4797-b976-31ae9844452c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:43.373713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-b49b60f3-ff6a-4797-b976-31ae9844452c None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:43.374938 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b49b60f3-ff6a-4797-b976-31ae9844452c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:43.424347 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:43.436980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1167/2327] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:43.444575 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cbed1-02da-48c2-9ffa-04c64631ceb7 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:43.444859 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cbed1-02da-48c2-9ffa-04c64631ceb7 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:43.445131 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cbed1-02da-48c2-9ffa-04c64631ceb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:43.465010 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-769cbed1-02da-48c2-9ffa-04c64631ceb7 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:43.466791 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-769cbed1-02da-48c2-9ffa-04c64631ceb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:43.474708 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1161/2328] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:43.478878 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b60f3-ff6a-4797-b976-31ae9844452c None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:43.489090 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b60f3-ff6a-4797-b976-31ae9844452c None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:43.493559 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153182a-d8a2-4ed9-8c87-8f41094d9129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:43.494147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153182a-d8a2-4ed9-8c87-8f41094d9129 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:43.494430 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153182a-d8a2-4ed9-8c87-8f41094d9129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:43.494688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153182a-d8a2-4ed9-8c87-8f41094d9129 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:43.495352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153182a-d8a2-4ed9-8c87-8f41094d9129 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:03:43.495649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153182a-d8a2-4ed9-8c87-8f41094d9129 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:43.497235 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153182a-d8a2-4ed9-8c87-8f41094d9129 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:43.508489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1168/2329] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:03:43.512790 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769cbed1-02da-48c2-9ffa-04c64631ceb7 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:43.522122 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769cbed1-02da-48c2-9ffa-04c64631ceb7 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:43.527060 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89afbe-f6e2-477c-b38a-a9aeef78520a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:43.527662 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89afbe-f6e2-477c-b38a-a9aeef78520a admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:43.527970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89afbe-f6e2-477c-b38a-a9aeef78520a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:43.528723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89afbe-f6e2-477c-b38a-a9aeef78520a admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:43.529369 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89afbe-f6e2-477c-b38a-a9aeef78520a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:43.531333 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89afbe-f6e2-477c-b38a-a9aeef78520a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:43.559243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1162/2330] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:03:43 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:03:43.564383 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153182a-d8a2-4ed9-8c87-8f41094d9129 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:43.575348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153182a-d8a2-4ed9-8c87-8f41094d9129 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:43.635163 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76bda9-e059-4e4b-a446-1d26e173e319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:43.635805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76bda9-e059-4e4b-a446-1d26e173e319 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:43.636086 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76bda9-e059-4e4b-a446-1d26e173e319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:43.636360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76bda9-e059-4e4b-a446-1d26e173e319 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:43.637008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76bda9-e059-4e4b-a446-1d26e173e319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:43.639005 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76bda9-e059-4e4b-a446-1d26e173e319 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:43.661302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1169/2331] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:03:43 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:03:43.666296 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89afbe-f6e2-477c-b38a-a9aeef78520a admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:43.678225 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89afbe-f6e2-477c-b38a-a9aeef78520a admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:43.683026 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c822c3d-b7e3-4922-ba9c-2541268d5f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:43.683636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c822c3d-b7e3-4922-ba9c-2541268d5f4a admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:43.683915 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c822c3d-b7e3-4922-ba9c-2541268d5f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:43.684188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c822c3d-b7e3-4922-ba9c-2541268d5f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:43.685654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c822c3d-b7e3-4922-ba9c-2541268d5f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:43.687325 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c822c3d-b7e3-4922-ba9c-2541268d5f4a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:43.697916 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1163/2332] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:03:43.702935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76bda9-e059-4e4b-a446-1d26e173e319 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:43.713307 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76bda9-e059-4e4b-a446-1d26e173e319 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:43.718198 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b0e4d-bcc2-4285-a777-3aaf0a3a56bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:43.718975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b0e4d-bcc2-4285-a777-3aaf0a3a56bf admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:43.719256 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b0e4d-bcc2-4285-a777-3aaf0a3a56bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:43.719527 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b0e4d-bcc2-4285-a777-3aaf0a3a56bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:43.720174 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b0e4d-bcc2-4285-a777-3aaf0a3a56bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:43.721795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b0e4d-bcc2-4285-a777-3aaf0a3a56bf admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:43.734772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1170/2333] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:03:43.736954 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c822c3d-b7e3-4922-ba9c-2541268d5f4a admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:43.747593 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c822c3d-b7e3-4922-ba9c-2541268d5f4a admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:43.752132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72340ee4-f944-4bfc-b3ce-fbad107b1113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:43.752767 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72340ee4-f944-4bfc-b3ce-fbad107b1113 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:43.753042 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72340ee4-f944-4bfc-b3ce-fbad107b1113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:43.753311 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72340ee4-f944-4bfc-b3ce-fbad107b1113 admin admin] PATH_INFO: `/v3/projects/e442375f966d4d65a5a411dfa175238e/users/2b548c343ebd4a11bc85eba1b73991f9/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:43.769395 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72340ee4-f944-4bfc-b3ce-fbad107b1113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e442375f966d4d65a5a411dfa175238e, user_id=2b548c343ebd4a11bc85eba1b73991f9, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:43.772098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72340ee4-f944-4bfc-b3ce-fbad107b1113 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:43.785464 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1164/2334] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:03:43 2022] PUT /identity/v3/projects/e442375f966d4d65a5a411dfa175238e/users/2b548c343ebd4a11bc85eba1b73991f9/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:03:43.790636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf500b9-e4d9-4f20-a116-5cab9eeda210 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:43.790950 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf500b9-e4d9-4f20-a116-5cab9eeda210 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:43.791225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf500b9-e4d9-4f20-a116-5cab9eeda210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:43.811338 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-dbf500b9-e4d9-4f20-a116-5cab9eeda210 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:43.812531 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-dbf500b9-e4d9-4f20-a116-5cab9eeda210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:43.870963 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1171/2335] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:43.881179 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72340ee4-f944-4bfc-b3ce-fbad107b1113 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:43.882922 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-72340ee4-f944-4bfc-b3ce-fbad107b1113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:43.925982 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:43.942201 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72340ee4-f944-4bfc-b3ce-fbad107b1113 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:43.946472 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1628bb7-3b94-43ee-8884-e0716accc272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:43.947155 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1628bb7-3b94-43ee-8884-e0716accc272 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:43.947462 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1628bb7-3b94-43ee-8884-e0716accc272 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:43.947734 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1628bb7-3b94-43ee-8884-e0716accc272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:43.957196 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1628bb7-3b94-43ee-8884-e0716accc272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:43.958685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1628bb7-3b94-43ee-8884-e0716accc272 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:43.969293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1165/2336] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:44.708453 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98b77d-94e6-4853-b383-58c9f1e9ea62 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:44.708991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98b77d-94e6-4853-b383-58c9f1e9ea62 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:44.708991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98b77d-94e6-4853-b383-58c9f1e9ea62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:44.739108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-dc98b77d-94e6-4853-b383-58c9f1e9ea62 None None] MFA Rules not processed for user `2b548c343ebd4a11bc85eba1b73991f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:44.740318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-dc98b77d-94e6-4853-b383-58c9f1e9ea62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:44.817238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:44.833637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1172/2337] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:03:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:03:44.838050 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1628bb7-3b94-43ee-8884-e0716accc272 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:44.849339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1628bb7-3b94-43ee-8884-e0716accc272 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:44.859153 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc059ce-b4b3-4c66-a084-64ec19b39adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:44.859787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc059ce-b4b3-4c66-a084-64ec19b39adb admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:44.860287 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc059ce-b4b3-4c66-a084-64ec19b39adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:44.860761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc059ce-b4b3-4c66-a084-64ec19b39adb admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:44.861475 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc059ce-b4b3-4c66-a084-64ec19b39adb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:44.863429 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc059ce-b4b3-4c66-a084-64ec19b39adb admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:44.899554 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1166/2338] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:03:44 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:03:44.904330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98b77d-94e6-4853-b383-58c9f1e9ea62 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:44.917170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98b77d-94e6-4853-b383-58c9f1e9ea62 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:44.980658 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30ca56-70ae-45c0-a294-7c2a84fa0521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:44.981290 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30ca56-70ae-45c0-a294-7c2a84fa0521 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:44.981576 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30ca56-70ae-45c0-a294-7c2a84fa0521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:44.981843 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30ca56-70ae-45c0-a294-7c2a84fa0521 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:44.982472 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30ca56-70ae-45c0-a294-7c2a84fa0521 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:44.984424 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30ca56-70ae-45c0-a294-7c2a84fa0521 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:45.006567 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1173/2339] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:03:44 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:03:45.011071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc059ce-b4b3-4c66-a084-64ec19b39adb admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:45.023476 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc059ce-b4b3-4c66-a084-64ec19b39adb admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:45.028628 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34a088-59c8-49a2-8186-d5957895019b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:45.029237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34a088-59c8-49a2-8186-d5957895019b admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:45.029592 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34a088-59c8-49a2-8186-d5957895019b admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:45.029788 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb34a088-59c8-49a2-8186-d5957895019b admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:45.030433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34a088-59c8-49a2-8186-d5957895019b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:45.032081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb34a088-59c8-49a2-8186-d5957895019b admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:45.043081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1167/2340] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:03:45.047497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30ca56-70ae-45c0-a294-7c2a84fa0521 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:45.058806 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30ca56-70ae-45c0-a294-7c2a84fa0521 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:45.064441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e6c82-4e9b-43c6-bfbd-c36e0c89e463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:45.065217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e6c82-4e9b-43c6-bfbd-c36e0c89e463 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:45.065528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e6c82-4e9b-43c6-bfbd-c36e0c89e463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:45.065965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e6c82-4e9b-43c6-bfbd-c36e0c89e463 admin admin] PATH_INFO: `/v3/projects/e4809c64f65e474e8f4fd0a7042d0579/users/135791cfaee3455486bf6d97132ac6da/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:45.082152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e6c82-4e9b-43c6-bfbd-c36e0c89e463 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4809c64f65e474e8f4fd0a7042d0579, user_id=135791cfaee3455486bf6d97132ac6da, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:45.084834 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e6c82-4e9b-43c6-bfbd-c36e0c89e463 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:45.099738 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1174/2341] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:03:45 2022] PUT /identity/v3/projects/e4809c64f65e474e8f4fd0a7042d0579/users/135791cfaee3455486bf6d97132ac6da/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:03:45.103793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34a088-59c8-49a2-8186-d5957895019b admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:45.114352 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb34a088-59c8-49a2-8186-d5957895019b admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:45.169990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab06d32-fb97-49b6-8697-0fca07d00f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:45.170595 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab06d32-fb97-49b6-8697-0fca07d00f9d admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:45.170906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab06d32-fb97-49b6-8697-0fca07d00f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:45.171179 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab06d32-fb97-49b6-8697-0fca07d00f9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:45.171817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab06d32-fb97-49b6-8697-0fca07d00f9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:45.173409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab06d32-fb97-49b6-8697-0fca07d00f9d admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:45.183043 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1168/2342] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:03:45.188319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e6c82-4e9b-43c6-bfbd-c36e0c89e463 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:45.199162 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e6c82-4e9b-43c6-bfbd-c36e0c89e463 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:45.204671 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84796989-8a3c-4d00-bbe6-6c10654d9218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:45.205335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84796989-8a3c-4d00-bbe6-6c10654d9218 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:45.205617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84796989-8a3c-4d00-bbe6-6c10654d9218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:45.205886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84796989-8a3c-4d00-bbe6-6c10654d9218 admin admin] PATH_INFO: `/v3/projects/e4809c64f65e474e8f4fd0a7042d0579/users/135791cfaee3455486bf6d97132ac6da/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:45.207814 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84796989-8a3c-4d00-bbe6-6c10654d9218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4809c64f65e474e8f4fd0a7042d0579, user_id=135791cfaee3455486bf6d97132ac6da, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:45.210443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84796989-8a3c-4d00-bbe6-6c10654d9218 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:45.221094 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1175/2343] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:03:45 2022] PUT /identity/v3/projects/e4809c64f65e474e8f4fd0a7042d0579/users/135791cfaee3455486bf6d97132ac6da/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:03:45.881829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeffeb3-da7b-4600-813b-f593dfa9859f None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:45.882275 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeffeb3-da7b-4600-813b-f593dfa9859f None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:45.882631 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeffeb3-da7b-4600-813b-f593dfa9859f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:45.914746 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-ceeffeb3-da7b-4600-813b-f593dfa9859f None None] MFA Rules not processed for user `135791cfaee3455486bf6d97132ac6da`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:03:45.915973 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ceeffeb3-da7b-4600-813b-f593dfa9859f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:45.990962 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1169/2344] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:03:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:03:46.002149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84796989-8a3c-4d00-bbe6-6c10654d9218 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:46.004044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-84796989-8a3c-4d00-bbe6-6c10654d9218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:46.068399 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84796989-8a3c-4d00-bbe6-6c10654d9218 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:46.072874 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2d5e3-206e-4c25-85cd-df24623c8752 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:46.073496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2d5e3-206e-4c25-85cd-df24623c8752 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:46.073774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2d5e3-206e-4c25-85cd-df24623c8752 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:46.074049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2d5e3-206e-4c25-85cd-df24623c8752 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:46.134307 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2d5e3-206e-4c25-85cd-df24623c8752 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:46.135780 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2d5e3-206e-4c25-85cd-df24623c8752 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:46.141048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:46.146770 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1176/2345] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:47.541283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeffeb3-da7b-4600-813b-f593dfa9859f None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:47.542783 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ceeffeb3-da7b-4600-813b-f593dfa9859f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:47.554709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeffeb3-da7b-4600-813b-f593dfa9859f None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:47.558356 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869df5a0-228a-43f0-8687-97e6dbd67aae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:47.559001 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869df5a0-228a-43f0-8687-97e6dbd67aae service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:47.559898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869df5a0-228a-43f0-8687-97e6dbd67aae service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:47.560022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869df5a0-228a-43f0-8687-97e6dbd67aae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:47.572192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869df5a0-228a-43f0-8687-97e6dbd67aae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:47.573706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869df5a0-228a-43f0-8687-97e6dbd67aae service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:47.585916 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1170/2346] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:47.611479 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2d5e3-206e-4c25-85cd-df24623c8752 service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:03:47.613044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-96c2d5e3-206e-4c25-85cd-df24623c8752 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:47.712113 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2d5e3-206e-4c25-85cd-df24623c8752 service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:03:47.720761 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775de98-e9f4-49f4-ab03-8ba2fa24a92b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:03:47.721492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775de98-e9f4-49f4-ab03-8ba2fa24a92b service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:03:47.721852 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775de98-e9f4-49f4-ab03-8ba2fa24a92b service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:03:47.722187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775de98-e9f4-49f4-ab03-8ba2fa24a92b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:03:47.724406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c775de98-e9f4-49f4-ab03-8ba2fa24a92b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:03:47.740751 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c775de98-e9f4-49f4-ab03-8ba2fa24a92b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:03:47.742527 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c775de98-e9f4-49f4-ab03-8ba2fa24a92b service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:03:47.754764 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1177/2347] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:03:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:03:57.580223 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:03:57.750041 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=34 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:00.791303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869df5a0-228a-43f0-8687-97e6dbd67aae service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:00.877813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869df5a0-228a-43f0-8687-97e6dbd67aae service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:00.882967 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09c082-05f9-4cbe-ac2a-30dc39ac90ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:00.883892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09c082-05f9-4cbe-ac2a-30dc39ac90ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:00.884253 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09c082-05f9-4cbe-ac2a-30dc39ac90ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:00.884588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09c082-05f9-4cbe-ac2a-30dc39ac90ec admin admin] PATH_INFO: `/v3/users/2b548c343ebd4a11bc85eba1b73991f9` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:00.885785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09c082-05f9-4cbe-ac2a-30dc39ac90ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b548c343ebd4a11bc85eba1b73991f9)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:00.887808 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc09c082-05f9-4cbe-ac2a-30dc39ac90ec admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:00.939611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-dc09c082-05f9-4cbe-ac2a-30dc39ac90ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b548c343ebd4a11bc85eba1b73991f9'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:00.951165 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-dc09c082-05f9-4cbe-ac2a-30dc39ac90ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b548c343ebd4a11bc85eba1b73991f9'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:00.951165 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-dc09c082-05f9-4cbe-ac2a-30dc39ac90ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b548c343ebd4a11bc85eba1b73991f9'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:00.966385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-dc09c082-05f9-4cbe-ac2a-30dc39ac90ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b548c343ebd4a11bc85eba1b73991f9'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:00.967169 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-dc09c082-05f9-4cbe-ac2a-30dc39ac90ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b548c343ebd4a11bc85eba1b73991f9'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:00.979535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1171/2348] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:04:00 2022] DELETE /identity/v3/users/2b548c343ebd4a11bc85eba1b73991f9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:01.308014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775de98-e9f4-49f4-ab03-8ba2fa24a92b service nova] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:01.318559 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775de98-e9f4-49f4-ab03-8ba2fa24a92b service nova] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:01.395921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec0698-9865-450d-b932-1dafdd1ec21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:01.396590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec0698-9865-450d-b932-1dafdd1ec21c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:01.396871 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec0698-9865-450d-b932-1dafdd1ec21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:01.397144 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec0698-9865-450d-b932-1dafdd1ec21c admin admin] PATH_INFO: `/v3/projects/e442375f966d4d65a5a411dfa175238e` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:01.398277 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec0698-9865-450d-b932-1dafdd1ec21c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e442375f966d4d65a5a411dfa175238e)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:01.400327 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec0698-9865-450d-b932-1dafdd1ec21c admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:01.462573 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f5ec0698-9865-450d-b932-1dafdd1ec21c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e442375f966d4d65a5a411dfa175238e', 'request_id': 'req-f5ec0698-9865-450d-b932-1dafdd1ec21c'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:01.469814 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f5ec0698-9865-450d-b932-1dafdd1ec21c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e442375f966d4d65a5a411dfa175238e', 'request_id': 'req-f5ec0698-9865-450d-b932-1dafdd1ec21c'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:01.476224 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f5ec0698-9865-450d-b932-1dafdd1ec21c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e442375f966d4d65a5a411dfa175238e', 'request_id': 'req-f5ec0698-9865-450d-b932-1dafdd1ec21c'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:01.493556 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f5ec0698-9865-450d-b932-1dafdd1ec21c admin admin] The token cache is being invalidate because project e442375f966d4d65a5a411dfa175238e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:04:01.494013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f5ec0698-9865-450d-b932-1dafdd1ec21c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:01.494826 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f5ec0698-9865-450d-b932-1dafdd1ec21c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:01.499910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1178/2349] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:04:01 2022] DELETE /identity/v3/projects/e442375f966d4d65a5a411dfa175238e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:01.501534 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09c082-05f9-4cbe-ac2a-30dc39ac90ec admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:01.503112 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-dc09c082-05f9-4cbe-ac2a-30dc39ac90ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:01.519138 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc09c082-05f9-4cbe-ac2a-30dc39ac90ec admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:01.640348 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f97480-bfe0-46b5-9dcb-d3d00a821838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:01.642883 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f97480-bfe0-46b5-9dcb-d3d00a821838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:01.643468 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f97480-bfe0-46b5-9dcb-d3d00a821838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:01.643919 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f97480-bfe0-46b5-9dcb-d3d00a821838 admin admin] PATH_INFO: `/v3/users/135791cfaee3455486bf6d97132ac6da` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:01.645594 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f97480-bfe0-46b5-9dcb-d3d00a821838 admin admin] RBAC: Authorizing `identity:delete_user(user_id=135791cfaee3455486bf6d97132ac6da)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:01.647906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f97480-bfe0-46b5-9dcb-d3d00a821838 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:01.682929 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:01.726659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f7f97480-bfe0-46b5-9dcb-d3d00a821838 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '135791cfaee3455486bf6d97132ac6da'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:01.737090 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f7f97480-bfe0-46b5-9dcb-d3d00a821838 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '135791cfaee3455486bf6d97132ac6da'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:01.738012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f7f97480-bfe0-46b5-9dcb-d3d00a821838 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '135791cfaee3455486bf6d97132ac6da'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:01.756263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f7f97480-bfe0-46b5-9dcb-d3d00a821838 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '135791cfaee3455486bf6d97132ac6da'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:01.757029 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f7f97480-bfe0-46b5-9dcb-d3d00a821838 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '135791cfaee3455486bf6d97132ac6da'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:01.772072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1172/2350] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:04:01 2022] DELETE /identity/v3/users/135791cfaee3455486bf6d97132ac6da => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:01.963124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec0698-9865-450d-b932-1dafdd1ec21c admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:01.964760 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f5ec0698-9865-450d-b932-1dafdd1ec21c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:01.980589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec0698-9865-450d-b932-1dafdd1ec21c admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:02.045507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35588901-f44c-47d5-b60b-022ec5b6f00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:02.046161 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35588901-f44c-47d5-b60b-022ec5b6f00e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:02.046630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35588901-f44c-47d5-b60b-022ec5b6f00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:02.047322 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35588901-f44c-47d5-b60b-022ec5b6f00e admin admin] PATH_INFO: `/v3/projects/e4809c64f65e474e8f4fd0a7042d0579` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:02.050370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35588901-f44c-47d5-b60b-022ec5b6f00e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4809c64f65e474e8f4fd0a7042d0579)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:02.053649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35588901-f44c-47d5-b60b-022ec5b6f00e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:02.113775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:02.131641 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-35588901-f44c-47d5-b60b-022ec5b6f00e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4809c64f65e474e8f4fd0a7042d0579', 'request_id': 'req-35588901-f44c-47d5-b60b-022ec5b6f00e'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:02.138265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-35588901-f44c-47d5-b60b-022ec5b6f00e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4809c64f65e474e8f4fd0a7042d0579', 'request_id': 'req-35588901-f44c-47d5-b60b-022ec5b6f00e'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:02.145890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-35588901-f44c-47d5-b60b-022ec5b6f00e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4809c64f65e474e8f4fd0a7042d0579', 'request_id': 'req-35588901-f44c-47d5-b60b-022ec5b6f00e'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:02.166423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-35588901-f44c-47d5-b60b-022ec5b6f00e admin admin] The token cache is being invalidate because project e4809c64f65e474e8f4fd0a7042d0579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:04:02.167024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-35588901-f44c-47d5-b60b-022ec5b6f00e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:02.168065 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-35588901-f44c-47d5-b60b-022ec5b6f00e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:02.171745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1179/2351] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:04:01 2022] DELETE /identity/v3/projects/e4809c64f65e474e8f4fd0a7042d0579 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:04:02.176334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985c443-1dc7-40f2-9a26-ddaff8c6dbbb None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:02.176811 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985c443-1dc7-40f2-9a26-ddaff8c6dbbb None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:02.176893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8985c443-1dc7-40f2-9a26-ddaff8c6dbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:02.198284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-8985c443-1dc7-40f2-9a26-ddaff8c6dbbb None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:02.199482 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-8985c443-1dc7-40f2-9a26-ddaff8c6dbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:02.277812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1173/2352] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:02.286670 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5e4f8-186e-4ad1-bc1a-47f3494f2470 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:02.286994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5e4f8-186e-4ad1-bc1a-47f3494f2470 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:02.287273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5e4f8-186e-4ad1-bc1a-47f3494f2470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:02.308611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-ebb5e4f8-186e-4ad1-bc1a-47f3494f2470 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:02.309970 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-ebb5e4f8-186e-4ad1-bc1a-47f3494f2470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:02.318607 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1180/2353] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:02.323066 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985c443-1dc7-40f2-9a26-ddaff8c6dbbb None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:02.335883 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8985c443-1dc7-40f2-9a26-ddaff8c6dbbb None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:02.340779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a975907-fca0-4993-8546-d49dff32a53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:02.341959 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a975907-fca0-4993-8546-d49dff32a53b admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:02.342238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a975907-fca0-4993-8546-d49dff32a53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:02.342502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a975907-fca0-4993-8546-d49dff32a53b admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:02.343189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a975907-fca0-4993-8546-d49dff32a53b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:04:02.343491 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a975907-fca0-4993-8546-d49dff32a53b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:02.345091 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a975907-fca0-4993-8546-d49dff32a53b admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:02.358282 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1174/2354] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:04:02.362833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5e4f8-186e-4ad1-bc1a-47f3494f2470 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:02.372764 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5e4f8-186e-4ad1-bc1a-47f3494f2470 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:02.377628 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408e540-c5e6-423b-8bc8-51a061feef0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:02.378239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408e540-c5e6-423b-8bc8-51a061feef0e admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:02.378519 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408e540-c5e6-423b-8bc8-51a061feef0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:02.378830 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408e540-c5e6-423b-8bc8-51a061feef0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:02.379469 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b408e540-c5e6-423b-8bc8-51a061feef0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:02.381379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b408e540-c5e6-423b-8bc8-51a061feef0e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:02.416225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1181/2355] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:04:02 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:04:02.421066 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a975907-fca0-4993-8546-d49dff32a53b admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:02.432122 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a975907-fca0-4993-8546-d49dff32a53b admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:02.498944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee8193-f503-4610-8b9b-3aada9939834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:02.499459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee8193-f503-4610-8b9b-3aada9939834 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:02.499735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee8193-f503-4610-8b9b-3aada9939834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:02.500002 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee8193-f503-4610-8b9b-3aada9939834 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:02.500643 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee8193-f503-4610-8b9b-3aada9939834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:02.502588 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee8193-f503-4610-8b9b-3aada9939834 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:02.529353 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1175/2356] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:04:02 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:04:02.531250 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408e540-c5e6-423b-8bc8-51a061feef0e admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:02.543117 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408e540-c5e6-423b-8bc8-51a061feef0e admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:02.553091 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3d912-8542-4768-b932-c720b565a8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:02.554445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3d912-8542-4768-b932-c720b565a8bd admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:02.554806 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3d912-8542-4768-b932-c720b565a8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:02.555087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3d912-8542-4768-b932-c720b565a8bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:02.555739 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3d912-8542-4768-b932-c720b565a8bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:02.557914 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3d912-8542-4768-b932-c720b565a8bd admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:02.572260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1182/2357] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:04:02.576052 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee8193-f503-4610-8b9b-3aada9939834 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:02.590584 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee8193-f503-4610-8b9b-3aada9939834 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:02.595990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b748773-1a22-4a76-9b20-45bc4525eebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:02.596665 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b748773-1a22-4a76-9b20-45bc4525eebe admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:02.596939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b748773-1a22-4a76-9b20-45bc4525eebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:02.597210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b748773-1a22-4a76-9b20-45bc4525eebe admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:02.597991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b748773-1a22-4a76-9b20-45bc4525eebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:02.600027 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b748773-1a22-4a76-9b20-45bc4525eebe admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:02.615392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1176/2358] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:04:02.616739 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3d912-8542-4768-b932-c720b565a8bd admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:02.627601 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3d912-8542-4768-b932-c720b565a8bd admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:02.632673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e7ab6-940f-4aa0-9658-816d78808090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:02.633477 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e7ab6-940f-4aa0-9658-816d78808090 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:02.633826 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e7ab6-940f-4aa0-9658-816d78808090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:02.634176 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e7ab6-940f-4aa0-9658-816d78808090 admin admin] PATH_INFO: `/v3/projects/c23e25b3ee2249dca0da09f0c431d26f/users/1fc2ba81e51f42ee91dda39d0120e594/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:02.652871 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e7ab6-940f-4aa0-9658-816d78808090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c23e25b3ee2249dca0da09f0c431d26f, user_id=1fc2ba81e51f42ee91dda39d0120e594, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:02.655783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e7ab6-940f-4aa0-9658-816d78808090 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:02.672813 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1183/2359] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:04:02 2022] PUT /identity/v3/projects/c23e25b3ee2249dca0da09f0c431d26f/users/1fc2ba81e51f42ee91dda39d0120e594/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:02.686004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9263d0-2b66-4175-a8dd-199271c9566b None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:02.686434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9263d0-2b66-4175-a8dd-199271c9566b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:02.686843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9263d0-2b66-4175-a8dd-199271c9566b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:02.702213 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:02.732882 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-bc9263d0-2b66-4175-a8dd-199271c9566b None None] MFA Rules not processed for user `1fc2ba81e51f42ee91dda39d0120e594`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:02.736025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-bc9263d0-2b66-4175-a8dd-199271c9566b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:02.839657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1177/2360] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:02.852097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e7ab6-940f-4aa0-9658-816d78808090 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:02.853901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1e9e7ab6-940f-4aa0-9658-816d78808090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:02.955277 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e7ab6-940f-4aa0-9658-816d78808090 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:02.959123 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08959233-30cc-4021-9e9c-4d81344bde6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:02.959810 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08959233-30cc-4021-9e9c-4d81344bde6f service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:02.960159 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08959233-30cc-4021-9e9c-4d81344bde6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:02.960785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08959233-30cc-4021-9e9c-4d81344bde6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:02.969940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08959233-30cc-4021-9e9c-4d81344bde6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:02.971415 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08959233-30cc-4021-9e9c-4d81344bde6f service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:02.980334 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1184/2361] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:04:03.002665 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9263d0-2b66-4175-a8dd-199271c9566b None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:03.069130 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9263d0-2b66-4175-a8dd-199271c9566b None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:03.075669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7bfa9-87b0-4ac9-9ef1-b32f75a29e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:03.076194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7bfa9-87b0-4ac9-9ef1-b32f75a29e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:03.076475 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7bfa9-87b0-4ac9-9ef1-b32f75a29e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:03.076751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7bfa9-87b0-4ac9-9ef1-b32f75a29e06 admin admin] PATH_INFO: `/v3/users/1fc2ba81e51f42ee91dda39d0120e594` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:03.077914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7bfa9-87b0-4ac9-9ef1-b32f75a29e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fc2ba81e51f42ee91dda39d0120e594)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:03.079875 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7bfa9-87b0-4ac9-9ef1-b32f75a29e06 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:03.136005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3da7bfa9-87b0-4ac9-9ef1-b32f75a29e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fc2ba81e51f42ee91dda39d0120e594'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:03.150457 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3da7bfa9-87b0-4ac9-9ef1-b32f75a29e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fc2ba81e51f42ee91dda39d0120e594'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:03.151292 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3da7bfa9-87b0-4ac9-9ef1-b32f75a29e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fc2ba81e51f42ee91dda39d0120e594'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:03.168817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3da7bfa9-87b0-4ac9-9ef1-b32f75a29e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fc2ba81e51f42ee91dda39d0120e594'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:03.170196 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3da7bfa9-87b0-4ac9-9ef1-b32f75a29e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fc2ba81e51f42ee91dda39d0120e594'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:03.188049 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1178/2362] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:04:03 2022] DELETE /identity/v3/users/1fc2ba81e51f42ee91dda39d0120e594 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:03.192237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11aa2c5-c2b1-49f0-91d6-2aa713cb4aca None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:03.192728 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11aa2c5-c2b1-49f0-91d6-2aa713cb4aca None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:03.193073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11aa2c5-c2b1-49f0-91d6-2aa713cb4aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:03.197819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22523483-01fc-4e87-b8d5-ebb355168919 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:03.198187 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22523483-01fc-4e87-b8d5-ebb355168919 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:03.198529 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22523483-01fc-4e87-b8d5-ebb355168919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:03.215621 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-c11aa2c5-c2b1-49f0-91d6-2aa713cb4aca None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:03.217607 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c11aa2c5-c2b1-49f0-91d6-2aa713cb4aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:03.222134 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-22523483-01fc-4e87-b8d5-ebb355168919 None None] MFA Rules not processed for user `6256407d9df648d5bbe33495cb9a8b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:03.223798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-22523483-01fc-4e87-b8d5-ebb355168919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:03.233743 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:03.281801 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:03.293020 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1185/2363] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:04:03.295801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1179/2364] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Sep 22 15:04:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:04:03.315068 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11aa2c5-c2b1-49f0-91d6-2aa713cb4aca None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:03.317800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c11aa2c5-c2b1-49f0-91d6-2aa713cb4aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:03.378156 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22523483-01fc-4e87-b8d5-ebb355168919 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:03.379621 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-22523483-01fc-4e87-b8d5-ebb355168919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:03.408807 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11aa2c5-c2b1-49f0-91d6-2aa713cb4aca None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:03.414680 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60c591-91a5-4188-90e8-39c95f6a0b1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:03.415519 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60c591-91a5-4188-90e8-39c95f6a0b1f service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:03.415807 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60c591-91a5-4188-90e8-39c95f6a0b1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:03.416079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60c591-91a5-4188-90e8-39c95f6a0b1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:03.427700 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60c591-91a5-4188-90e8-39c95f6a0b1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:03.429571 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60c591-91a5-4188-90e8-39c95f6a0b1f service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:03.442173 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1186/2365] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:04:03.460684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22523483-01fc-4e87-b8d5-ebb355168919 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:03.465966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da11b2-a6a8-4cee-8a45-91ded006e28d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a265623ae734936a340c97df852e227', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:03.466581 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da11b2-a6a8-4cee-8a45-91ded006e28d service swift] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:03.466909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da11b2-a6a8-4cee-8a45-91ded006e28d service swift] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:03.467190 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da11b2-a6a8-4cee-8a45-91ded006e28d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:03.475757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da11b2-a6a8-4cee-8a45-91ded006e28d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:03.477275 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da11b2-a6a8-4cee-8a45-91ded006e28d service swift] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:03.485923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1180/2366] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Sep 22 15:04:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:04:04.096056 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60c591-91a5-4188-90e8-39c95f6a0b1f service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:04.097892 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-dd60c591-91a5-4188-90e8-39c95f6a0b1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:04.119385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60c591-91a5-4188-90e8-39c95f6a0b1f service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:04.129482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193ecd4-9a94-4159-bf5b-1ac258a259fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:04.130312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193ecd4-9a94-4159-bf5b-1ac258a259fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:04.130747 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193ecd4-9a94-4159-bf5b-1ac258a259fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:04.139798 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193ecd4-9a94-4159-bf5b-1ac258a259fa admin admin] PATH_INFO: `/v3/projects/c23e25b3ee2249dca0da09f0c431d26f` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:04.143447 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193ecd4-9a94-4159-bf5b-1ac258a259fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c23e25b3ee2249dca0da09f0c431d26f)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:04.148482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193ecd4-9a94-4159-bf5b-1ac258a259fa admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:04.238507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c193ecd4-9a94-4159-bf5b-1ac258a259fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c23e25b3ee2249dca0da09f0c431d26f', 'request_id': 'req-c193ecd4-9a94-4159-bf5b-1ac258a259fa'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:04.251353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c193ecd4-9a94-4159-bf5b-1ac258a259fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c23e25b3ee2249dca0da09f0c431d26f', 'request_id': 'req-c193ecd4-9a94-4159-bf5b-1ac258a259fa'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:04.263268 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c193ecd4-9a94-4159-bf5b-1ac258a259fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c23e25b3ee2249dca0da09f0c431d26f', 'request_id': 'req-c193ecd4-9a94-4159-bf5b-1ac258a259fa'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:04.293068 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c193ecd4-9a94-4159-bf5b-1ac258a259fa admin admin] The token cache is being invalidate because project c23e25b3ee2249dca0da09f0c431d26f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:04:04.293372 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c193ecd4-9a94-4159-bf5b-1ac258a259fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:04.294526 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c193ecd4-9a94-4159-bf5b-1ac258a259fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:04.300378 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1187/2367] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:04:04 2022] DELETE /identity/v3/projects/c23e25b3ee2249dca0da09f0c431d26f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:04.301938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998ed19-6940-40fa-a6e4-9c96dde5fc93 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:04.302427 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998ed19-6940-40fa-a6e4-9c96dde5fc93 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:04.302630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998ed19-6940-40fa-a6e4-9c96dde5fc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:04.345903 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-9998ed19-6940-40fa-a6e4-9c96dde5fc93 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:04.348070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-9998ed19-6940-40fa-a6e4-9c96dde5fc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:04.474797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1181/2368] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:04.484093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d3a82-fc8e-41bf-95e8-254805714cc4 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:04.484578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d3a82-fc8e-41bf-95e8-254805714cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:04.484678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d3a82-fc8e-41bf-95e8-254805714cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:04.513479 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-ca1d3a82-fc8e-41bf-95e8-254805714cc4 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:04.515074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-ca1d3a82-fc8e-41bf-95e8-254805714cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:04.529490 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998ed19-6940-40fa-a6e4-9c96dde5fc93 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:04.530069 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1188/2369] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:04.545373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998ed19-6940-40fa-a6e4-9c96dde5fc93 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:04.557589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce881455-9354-4544-bdee-c483e4c2ceed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:04.558421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce881455-9354-4544-bdee-c483e4c2ceed admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:04.558839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce881455-9354-4544-bdee-c483e4c2ceed admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:04.560101 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce881455-9354-4544-bdee-c483e4c2ceed admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:04.560921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce881455-9354-4544-bdee-c483e4c2ceed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:04:04.561925 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce881455-9354-4544-bdee-c483e4c2ceed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:04.566198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce881455-9354-4544-bdee-c483e4c2ceed admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:04.600117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1182/2370] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 15:04:04.603086 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d3a82-fc8e-41bf-95e8-254805714cc4 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:04.616033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d3a82-fc8e-41bf-95e8-254805714cc4 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:04.636170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f99e49-e26f-4438-a7a7-c490f0381fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:04.637041 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f99e49-e26f-4438-a7a7-c490f0381fad admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:04.639128 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f99e49-e26f-4438-a7a7-c490f0381fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:04.639128 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f99e49-e26f-4438-a7a7-c490f0381fad admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:04.642882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f99e49-e26f-4438-a7a7-c490f0381fad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:04.645358 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f99e49-e26f-4438-a7a7-c490f0381fad admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:04.721698 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce881455-9354-4544-bdee-c483e4c2ceed admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:04.727007 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1189/2371] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:04:04 2022] POST /identity/v3/projects => generated 363 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:04:04.727985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f99e49-e26f-4438-a7a7-c490f0381fad admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:04.728297 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ce881455-9354-4544-bdee-c483e4c2ceed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:04.818264 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:04.849120 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:04.857313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f99e49-e26f-4438-a7a7-c490f0381fad admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:04.875655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584ec8f-4d3e-4fee-a354-23eff351228e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:04.876619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584ec8f-4d3e-4fee-a354-23eff351228e admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:04.877938 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584ec8f-4d3e-4fee-a354-23eff351228e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:04.878932 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584ec8f-4d3e-4fee-a354-23eff351228e admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:04.880649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584ec8f-4d3e-4fee-a354-23eff351228e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:04.889079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584ec8f-4d3e-4fee-a354-23eff351228e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:04.905242 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce881455-9354-4544-bdee-c483e4c2ceed admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:04.940030 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1190/2372] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:04:04 2022] POST /identity/v3/users => generated 282 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:04:04.941079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584ec8f-4d3e-4fee-a354-23eff351228e admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:04.952449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584ec8f-4d3e-4fee-a354-23eff351228e admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:04.960032 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1975eef-b885-42ae-8ce1-044ffce15c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:04.961062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1975eef-b885-42ae-8ce1-044ffce15c89 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:04.961469 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1975eef-b885-42ae-8ce1-044ffce15c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:04.961844 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1975eef-b885-42ae-8ce1-044ffce15c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:04.962627 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1975eef-b885-42ae-8ce1-044ffce15c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:04.965471 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1975eef-b885-42ae-8ce1-044ffce15c89 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:04.980251 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1191/2373] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:04:04.981343 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1975eef-b885-42ae-8ce1-044ffce15c89 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:04.991600 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1975eef-b885-42ae-8ce1-044ffce15c89 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:04.996674 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94734bc3-723f-45cc-a9a5-547b90c4f748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:04.997457 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94734bc3-723f-45cc-a9a5-547b90c4f748 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:04.997847 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94734bc3-723f-45cc-a9a5-547b90c4f748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:04.998349 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94734bc3-723f-45cc-a9a5-547b90c4f748 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:04.999176 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94734bc3-723f-45cc-a9a5-547b90c4f748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:05.000926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94734bc3-723f-45cc-a9a5-547b90c4f748 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:05.013254 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1192/2374] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:04:05.017552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94734bc3-723f-45cc-a9a5-547b90c4f748 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:05.034067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94734bc3-723f-45cc-a9a5-547b90c4f748 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:05.039488 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452a93a-c4fa-472c-b03f-2177401960b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:05.040694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452a93a-c4fa-472c-b03f-2177401960b9 service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:05.041630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452a93a-c4fa-472c-b03f-2177401960b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:05.042526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452a93a-c4fa-472c-b03f-2177401960b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:05.046215 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-9452a93a-c4fa-472c-b03f-2177401960b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:05.047980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfc390-3d0e-4cb8-a4d1-273e7341396d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:05.049068 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfc390-3d0e-4cb8-a4d1-273e7341396d admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:05.049499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfc390-3d0e-4cb8-a4d1-273e7341396d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:05.050209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bfc390-3d0e-4cb8-a4d1-273e7341396d admin admin] PATH_INFO: `/v3/projects/1d2bc66eec1c4b6895f18b1f05545d3c/users/b1b0919011234b15bdfe5680fdcc3a0b/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:05.071030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452a93a-c4fa-472c-b03f-2177401960b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:05.073089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452a93a-c4fa-472c-b03f-2177401960b9 service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:05.087774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bfc390-3d0e-4cb8-a4d1-273e7341396d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d2bc66eec1c4b6895f18b1f05545d3c, user_id=b1b0919011234b15bdfe5680fdcc3a0b, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:05.088459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1183/2375] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:04:05.100668 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bfc390-3d0e-4cb8-a4d1-273e7341396d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:05.130015 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1193/2376] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:04:05 2022] PUT /identity/v3/projects/1d2bc66eec1c4b6895f18b1f05545d3c/users/b1b0919011234b15bdfe5680fdcc3a0b/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:04:05.141391 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939337a-f768-40b1-964a-ab6547fe641d None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:05.141683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939337a-f768-40b1-964a-ab6547fe641d None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:05.141963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939337a-f768-40b1-964a-ab6547fe641d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:05.174523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-0939337a-f768-40b1-964a-ab6547fe641d None None] MFA Rules not processed for user `b1b0919011234b15bdfe5680fdcc3a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:05.176067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-0939337a-f768-40b1-964a-ab6547fe641d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:05.348684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1184/2377] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:05.361108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfc390-3d0e-4cb8-a4d1-273e7341396d admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:05.363442 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a6bfc390-3d0e-4cb8-a4d1-273e7341396d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:05.474401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bfc390-3d0e-4cb8-a4d1-273e7341396d admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:05.484590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1ba5e-2b5a-4ab6-a34e-27cbbd977b28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:05.485630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1ba5e-2b5a-4ab6-a34e-27cbbd977b28 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:05.485630 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1ba5e-2b5a-4ab6-a34e-27cbbd977b28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:05.486049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1ba5e-2b5a-4ab6-a34e-27cbbd977b28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:05.514716 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1ba5e-2b5a-4ab6-a34e-27cbbd977b28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:05.516807 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1ba5e-2b5a-4ab6-a34e-27cbbd977b28 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:05.547617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1194/2378] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:04:06.101601 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949ab96-16e9-4cfd-9920-e5970306b2c2 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:06.101889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949ab96-16e9-4cfd-9920-e5970306b2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:06.102161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949ab96-16e9-4cfd-9920-e5970306b2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:06.124907 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-6949ab96-16e9-4cfd-9920-e5970306b2c2 None None] MFA Rules not processed for user `da004a191bcd43b6a0865919813ec1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:06.126086 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-6949ab96-16e9-4cfd-9920-e5970306b2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:06.174548 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:06.187742 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1185/2379] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:04:06.560785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1ba5e-2b5a-4ab6-a34e-27cbbd977b28 service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:06.562345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7cd1ba5e-2b5a-4ab6-a34e-27cbbd977b28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:06.647582 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1ba5e-2b5a-4ab6-a34e-27cbbd977b28 service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:06.651921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30222d-bb1e-4b06-98e6-734359964564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:06.652719 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30222d-bb1e-4b06-98e6-734359964564 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:06.653124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30222d-bb1e-4b06-98e6-734359964564 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:06.654073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30222d-bb1e-4b06-98e6-734359964564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:06.656003 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-ef30222d-bb1e-4b06-98e6-734359964564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:06.670198 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30222d-bb1e-4b06-98e6-734359964564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:06.671821 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30222d-bb1e-4b06-98e6-734359964564 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:06.681905 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1195/2380] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:04:16.180793 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:16.676921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=75,keystone:INSERT=4 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:23.034661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949ab96-16e9-4cfd-9920-e5970306b2c2 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:23.036142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-6949ab96-16e9-4cfd-9920-e5970306b2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:23.129336 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949ab96-16e9-4cfd-9920-e5970306b2c2 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:23.134935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0280ef-a0cc-4284-a14e-4e2458780156 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:23.135554 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0280ef-a0cc-4284-a14e-4e2458780156 service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:23.135832 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0280ef-a0cc-4284-a14e-4e2458780156 service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:23.136107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0280ef-a0cc-4284-a14e-4e2458780156 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:23.138031 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3a0280ef-a0cc-4284-a14e-4e2458780156 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:23.154966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0280ef-a0cc-4284-a14e-4e2458780156 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:23.156574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0280ef-a0cc-4284-a14e-4e2458780156 service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:23.168421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1186/2381] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:04:23.343806 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b24253-c9a0-4194-9c77-63d68fd55898 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:23.344336 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b24253-c9a0-4194-9c77-63d68fd55898 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:23.344336 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b24253-c9a0-4194-9c77-63d68fd55898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:23.365371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-50b24253-c9a0-4194-9c77-63d68fd55898 None None] MFA Rules not processed for user `6256407d9df648d5bbe33495cb9a8b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:23.366585 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-50b24253-c9a0-4194-9c77-63d68fd55898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:23.437288 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1196/2382] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Sep 22 15:04:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:23.450064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0280ef-a0cc-4284-a14e-4e2458780156 service nova] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:23.452370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3a0280ef-a0cc-4284-a14e-4e2458780156 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:23.543639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0280ef-a0cc-4284-a14e-4e2458780156 service nova] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:23.558199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b42990-515f-4163-8ab9-f1791e2f799b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a265623ae734936a340c97df852e227', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:23.559257 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b42990-515f-4163-8ab9-f1791e2f799b service swift] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:23.559744 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b42990-515f-4163-8ab9-f1791e2f799b service swift] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:23.560013 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b42990-515f-4163-8ab9-f1791e2f799b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:23.574253 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b42990-515f-4163-8ab9-f1791e2f799b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:23.575973 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b42990-515f-4163-8ab9-f1791e2f799b service swift] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:23.586007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1187/2383] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Sep 22 15:04:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:04:33.429733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=25 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:33.583119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=66 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:40.327064 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b8e1e-dbb6-42ad-812e-f200d8844b8c None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:40.327582 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b8e1e-dbb6-42ad-812e-f200d8844b8c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:40.327582 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b8e1e-dbb6-42ad-812e-f200d8844b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:40.347542 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-ea2b8e1e-dbb6-42ad-812e-f200d8844b8c None None] MFA Rules not processed for user `6256407d9df648d5bbe33495cb9a8b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:40.348742 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-ea2b8e1e-dbb6-42ad-812e-f200d8844b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:40.355900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1197/2384] 10.222.0.26 () {58 vars in 967 bytes} [Thu Sep 22 15:04:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:04:40.369227 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b42990-515f-4163-8ab9-f1791e2f799b service swift] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:40.380775 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b42990-515f-4163-8ab9-f1791e2f799b service swift] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:40.386147 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe37a5-b45a-4e86-b8eb-bf2cacf37acf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a265623ae734936a340c97df852e227', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:40.386776 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe37a5-b45a-4e86-b8eb-bf2cacf37acf service swift] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:40.387059 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe37a5-b45a-4e86-b8eb-bf2cacf37acf service swift] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:40.387335 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe37a5-b45a-4e86-b8eb-bf2cacf37acf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:40.395746 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe37a5-b45a-4e86-b8eb-bf2cacf37acf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:40.397331 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe37a5-b45a-4e86-b8eb-bf2cacf37acf service swift] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:40.409485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1188/2385] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Sep 22 15:04:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:04:41.898793 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b8e1e-dbb6-42ad-812e-f200d8844b8c None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:41.974075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b8e1e-dbb6-42ad-812e-f200d8844b8c None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:41.979128 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a964830-f2ab-48da-bdfc-e9ef49145929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:41.979764 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a964830-f2ab-48da-bdfc-e9ef49145929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:41.980043 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a964830-f2ab-48da-bdfc-e9ef49145929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:41.980315 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a964830-f2ab-48da-bdfc-e9ef49145929 admin admin] PATH_INFO: `/v3/users/b1b0919011234b15bdfe5680fdcc3a0b` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:41.981426 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a964830-f2ab-48da-bdfc-e9ef49145929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1b0919011234b15bdfe5680fdcc3a0b)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:41.984150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a964830-f2ab-48da-bdfc-e9ef49145929 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:42.034421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6a964830-f2ab-48da-bdfc-e9ef49145929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1b0919011234b15bdfe5680fdcc3a0b'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:42.042058 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6a964830-f2ab-48da-bdfc-e9ef49145929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1b0919011234b15bdfe5680fdcc3a0b'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:42.051308 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6a964830-f2ab-48da-bdfc-e9ef49145929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1b0919011234b15bdfe5680fdcc3a0b'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:42.064482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6a964830-f2ab-48da-bdfc-e9ef49145929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1b0919011234b15bdfe5680fdcc3a0b'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:42.065208 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6a964830-f2ab-48da-bdfc-e9ef49145929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1b0919011234b15bdfe5680fdcc3a0b'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:42.071986 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1198/2386] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:04:41 2022] DELETE /identity/v3/users/b1b0919011234b15bdfe5680fdcc3a0b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:42.077071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2729e13-6f35-4642-bd78-d376d5149a31 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:42.077579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2729e13-6f35-4642-bd78-d376d5149a31 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:42.077715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2729e13-6f35-4642-bd78-d376d5149a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:42.100264 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-d2729e13-6f35-4642-bd78-d376d5149a31 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:42.101405 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d2729e13-6f35-4642-bd78-d376d5149a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:42.160264 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1189/2387] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:04:42.170572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a964830-f2ab-48da-bdfc-e9ef49145929 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:42.172016 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-6a964830-f2ab-48da-bdfc-e9ef49145929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:42.228257 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a964830-f2ab-48da-bdfc-e9ef49145929 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:42.232584 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04118a-7ab7-4031-b006-9bcb327014eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:42.233176 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04118a-7ab7-4031-b006-9bcb327014eb service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:42.233451 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04118a-7ab7-4031-b006-9bcb327014eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:42.233716 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04118a-7ab7-4031-b006-9bcb327014eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:42.244551 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04118a-7ab7-4031-b006-9bcb327014eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:42.246020 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04118a-7ab7-4031-b006-9bcb327014eb service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:42.256413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1199/2388] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:04:42.543637 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2729e13-6f35-4642-bd78-d376d5149a31 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:42.545401 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d2729e13-6f35-4642-bd78-d376d5149a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:42.565985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2729e13-6f35-4642-bd78-d376d5149a31 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:42.571210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efff80b-12e2-4593-9d90-f92f7d6a5e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:42.572395 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efff80b-12e2-4593-9d90-f92f7d6a5e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:42.572681 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efff80b-12e2-4593-9d90-f92f7d6a5e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:42.572959 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efff80b-12e2-4593-9d90-f92f7d6a5e5d admin admin] PATH_INFO: `/v3/projects/1d2bc66eec1c4b6895f18b1f05545d3c` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:42.574114 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efff80b-12e2-4593-9d90-f92f7d6a5e5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d2bc66eec1c4b6895f18b1f05545d3c)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:42.576591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efff80b-12e2-4593-9d90-f92f7d6a5e5d admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:42.625244 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5efff80b-12e2-4593-9d90-f92f7d6a5e5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d2bc66eec1c4b6895f18b1f05545d3c', 'request_id': 'req-5efff80b-12e2-4593-9d90-f92f7d6a5e5d'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:42.632642 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5efff80b-12e2-4593-9d90-f92f7d6a5e5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d2bc66eec1c4b6895f18b1f05545d3c', 'request_id': 'req-5efff80b-12e2-4593-9d90-f92f7d6a5e5d'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:42.639599 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5efff80b-12e2-4593-9d90-f92f7d6a5e5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d2bc66eec1c4b6895f18b1f05545d3c', 'request_id': 'req-5efff80b-12e2-4593-9d90-f92f7d6a5e5d'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:42.649692 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5efff80b-12e2-4593-9d90-f92f7d6a5e5d admin admin] The token cache is being invalidate because project 1d2bc66eec1c4b6895f18b1f05545d3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:04:42.649976 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5efff80b-12e2-4593-9d90-f92f7d6a5e5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:42.650600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5efff80b-12e2-4593-9d90-f92f7d6a5e5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:42.652113 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1190/2389] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:04:42 2022] DELETE /identity/v3/projects/1d2bc66eec1c4b6895f18b1f05545d3c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:42.657067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a6f52-1410-47a7-99fa-cc91006abd7a None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:42.657348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a6f52-1410-47a7-99fa-cc91006abd7a None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:42.657621 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a6f52-1410-47a7-99fa-cc91006abd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:42.676809 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-9e1a6f52-1410-47a7-99fa-cc91006abd7a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:42.678015 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-9e1a6f52-1410-47a7-99fa-cc91006abd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:42.687900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:42.738624 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1200/2390] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:42.746982 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c94806-045a-4308-8762-f1312bcc050e None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:42.747767 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c94806-045a-4308-8762-f1312bcc050e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:42.748043 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c94806-045a-4308-8762-f1312bcc050e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:42.768428 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-93c94806-045a-4308-8762-f1312bcc050e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:42.769644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-93c94806-045a-4308-8762-f1312bcc050e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:42.778194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1191/2391] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:42.782909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a6f52-1410-47a7-99fa-cc91006abd7a None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:42.792698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a6f52-1410-47a7-99fa-cc91006abd7a None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:42.797331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6248f-bdde-4978-8c4a-02b14f7c6263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:42.797926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6248f-bdde-4978-8c4a-02b14f7c6263 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:42.798205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6248f-bdde-4978-8c4a-02b14f7c6263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:42.798475 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6248f-bdde-4978-8c4a-02b14f7c6263 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:42.799149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6248f-bdde-4978-8c4a-02b14f7c6263 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:04:42.799450 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6248f-bdde-4978-8c4a-02b14f7c6263 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:42.801036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf6248f-bdde-4978-8c4a-02b14f7c6263 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:42.812947 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1201/2392] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:04:42.817249 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c94806-045a-4308-8762-f1312bcc050e None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:42.827191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c94806-045a-4308-8762-f1312bcc050e None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:42.831839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6cbf6-a0bf-4fff-881a-159c2e65c4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:42.832437 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6cbf6-a0bf-4fff-881a-159c2e65c4c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:42.832715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6cbf6-a0bf-4fff-881a-159c2e65c4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:42.832987 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6cbf6-a0bf-4fff-881a-159c2e65c4c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:42.833637 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6cbf6-a0bf-4fff-881a-159c2e65c4c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:42.835812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d6cbf6-a0bf-4fff-881a-159c2e65c4c5 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:42.866248 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1192/2393] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:04:42 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:04:42.869086 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6248f-bdde-4978-8c4a-02b14f7c6263 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:42.880049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6248f-bdde-4978-8c4a-02b14f7c6263 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:42.940100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac0c97-9081-45dd-a156-d5d3a1e70e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:42.940973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac0c97-9081-45dd-a156-d5d3a1e70e03 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:42.942166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac0c97-9081-45dd-a156-d5d3a1e70e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:42.942518 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac0c97-9081-45dd-a156-d5d3a1e70e03 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:42.943637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac0c97-9081-45dd-a156-d5d3a1e70e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:42.945628 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac0c97-9081-45dd-a156-d5d3a1e70e03 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:42.968897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1202/2394] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:04:42 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:04:42.973086 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6cbf6-a0bf-4fff-881a-159c2e65c4c5 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:42.984574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6cbf6-a0bf-4fff-881a-159c2e65c4c5 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:42.989385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a2f88-0fd8-4e03-83a3-cda08c9398fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:42.990012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a2f88-0fd8-4e03-83a3-cda08c9398fe admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:42.990285 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a2f88-0fd8-4e03-83a3-cda08c9398fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:42.990551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a2f88-0fd8-4e03-83a3-cda08c9398fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:42.991224 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a2f88-0fd8-4e03-83a3-cda08c9398fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:42.992836 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a2f88-0fd8-4e03-83a3-cda08c9398fe admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:43.003853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1193/2395] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:04:43.007904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac0c97-9081-45dd-a156-d5d3a1e70e03 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:43.017832 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac0c97-9081-45dd-a156-d5d3a1e70e03 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:43.026549 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab218da5-d30b-4c18-8203-d85724425254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:43.027311 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab218da5-d30b-4c18-8203-d85724425254 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:43.027597 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab218da5-d30b-4c18-8203-d85724425254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:43.027872 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab218da5-d30b-4c18-8203-d85724425254 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:43.028735 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab218da5-d30b-4c18-8203-d85724425254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:43.030174 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab218da5-d30b-4c18-8203-d85724425254 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:43.041703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1203/2396] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:04:43.047140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a2f88-0fd8-4e03-83a3-cda08c9398fe admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:43.060862 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a2f88-0fd8-4e03-83a3-cda08c9398fe admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:43.065759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f125d90-1224-472d-9418-b00122404f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:43.066510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f125d90-1224-472d-9418-b00122404f3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:43.066890 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f125d90-1224-472d-9418-b00122404f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:43.067230 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f125d90-1224-472d-9418-b00122404f3e admin admin] PATH_INFO: `/v3/projects/9898df110866463ab0c30c61d9d872f7/users/c232dca57ddc4ef496cf04c28dff4e5e/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:43.084416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f125d90-1224-472d-9418-b00122404f3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9898df110866463ab0c30c61d9d872f7, user_id=c232dca57ddc4ef496cf04c28dff4e5e, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:43.087574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f125d90-1224-472d-9418-b00122404f3e admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:43.101332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1194/2397] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:04:43 2022] PUT /identity/v3/projects/9898df110866463ab0c30c61d9d872f7/users/c232dca57ddc4ef496cf04c28dff4e5e/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:43.113364 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36764a5d-9761-4c48-92bf-80605e411deb None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:43.113364 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36764a5d-9761-4c48-92bf-80605e411deb None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:43.113364 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36764a5d-9761-4c48-92bf-80605e411deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:43.142820 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-36764a5d-9761-4c48-92bf-80605e411deb None None] MFA Rules not processed for user `c232dca57ddc4ef496cf04c28dff4e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:43.144168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-36764a5d-9761-4c48-92bf-80605e411deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:43.218305 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1204/2398] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:43.229126 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f125d90-1224-472d-9418-b00122404f3e admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:43.230975 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-9f125d90-1224-472d-9418-b00122404f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:43.241504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:43.291281 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f125d90-1224-472d-9418-b00122404f3e admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:43.295500 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef463d50-1bd4-4650-822a-564487286d41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:43.296110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef463d50-1bd4-4650-822a-564487286d41 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:43.296388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef463d50-1bd4-4650-822a-564487286d41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:43.296659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef463d50-1bd4-4650-822a-564487286d41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:43.305319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef463d50-1bd4-4650-822a-564487286d41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:43.306786 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef463d50-1bd4-4650-822a-564487286d41 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:43.317415 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1195/2399] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:04:51.080705 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36764a5d-9761-4c48-92bf-80605e411deb None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:51.082126 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-36764a5d-9761-4c48-92bf-80605e411deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:51.096195 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:51.140176 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36764a5d-9761-4c48-92bf-80605e411deb None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:51.143733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441c93c-e629-4629-9bee-11ed3210f017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:51.144337 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441c93c-e629-4629-9bee-11ed3210f017 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:51.144609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441c93c-e629-4629-9bee-11ed3210f017 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:51.144874 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441c93c-e629-4629-9bee-11ed3210f017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:51.146590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a441c93c-e629-4629-9bee-11ed3210f017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:51.209348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441c93c-e629-4629-9bee-11ed3210f017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:51.211096 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441c93c-e629-4629-9bee-11ed3210f017 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:51.221272 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1205/2400] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:04:53.312541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=23 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:53.648291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef463d50-1bd4-4650-822a-564487286d41 service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:53.663710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef463d50-1bd4-4650-822a-564487286d41 service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:53.668593 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a0a3a-9cc8-4715-8435-6fd3e03f12ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:53.670355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a0a3a-9cc8-4715-8435-6fd3e03f12ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:53.670355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a0a3a-9cc8-4715-8435-6fd3e03f12ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:53.670355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a0a3a-9cc8-4715-8435-6fd3e03f12ec admin admin] PATH_INFO: `/v3/users/c232dca57ddc4ef496cf04c28dff4e5e` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:53.671739 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a0a3a-9cc8-4715-8435-6fd3e03f12ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=c232dca57ddc4ef496cf04c28dff4e5e)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:53.673787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a0a3a-9cc8-4715-8435-6fd3e03f12ec admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:53.740301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-808a0a3a-9cc8-4715-8435-6fd3e03f12ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c232dca57ddc4ef496cf04c28dff4e5e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:53.748795 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-808a0a3a-9cc8-4715-8435-6fd3e03f12ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c232dca57ddc4ef496cf04c28dff4e5e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:53.749600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-808a0a3a-9cc8-4715-8435-6fd3e03f12ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c232dca57ddc4ef496cf04c28dff4e5e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:53.763354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-808a0a3a-9cc8-4715-8435-6fd3e03f12ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c232dca57ddc4ef496cf04c28dff4e5e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:53.764200 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-808a0a3a-9cc8-4715-8435-6fd3e03f12ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c232dca57ddc4ef496cf04c28dff4e5e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:53.777634 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1196/2401] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:04:53 2022] DELETE /identity/v3/users/c232dca57ddc4ef496cf04c28dff4e5e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:53.782272 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde5020-4b37-40e7-ac5a-6c8dee85a2b4 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:53.782839 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde5020-4b37-40e7-ac5a-6c8dee85a2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:53.783000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde5020-4b37-40e7-ac5a-6c8dee85a2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:53.804029 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-1bde5020-4b37-40e7-ac5a-6c8dee85a2b4 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:53.805376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1bde5020-4b37-40e7-ac5a-6c8dee85a2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:53.871356 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1206/2402] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:53.881244 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a0a3a-9cc8-4715-8435-6fd3e03f12ec admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:53.882658 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-808a0a3a-9cc8-4715-8435-6fd3e03f12ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:53.941583 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a0a3a-9cc8-4715-8435-6fd3e03f12ec admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:53.945267 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090062f2-24cf-43fa-9960-e95be2dd1680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:53.945989 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090062f2-24cf-43fa-9960-e95be2dd1680 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:53.946268 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090062f2-24cf-43fa-9960-e95be2dd1680 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:53.946542 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090062f2-24cf-43fa-9960-e95be2dd1680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:53.955962 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090062f2-24cf-43fa-9960-e95be2dd1680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:53.957395 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090062f2-24cf-43fa-9960-e95be2dd1680 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:53.967768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1197/2403] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:04:54.426506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde5020-4b37-40e7-ac5a-6c8dee85a2b4 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:54.428380 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1bde5020-4b37-40e7-ac5a-6c8dee85a2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:54.441503 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde5020-4b37-40e7-ac5a-6c8dee85a2b4 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:54.450531 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414ee28-2d49-4de7-94c4-853f7806f01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:54.451984 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414ee28-2d49-4de7-94c4-853f7806f01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:54.452132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414ee28-2d49-4de7-94c4-853f7806f01a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:54.452430 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414ee28-2d49-4de7-94c4-853f7806f01a admin admin] PATH_INFO: `/v3/projects/9898df110866463ab0c30c61d9d872f7` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:54.453820 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414ee28-2d49-4de7-94c4-853f7806f01a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9898df110866463ab0c30c61d9d872f7)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:54.456330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414ee28-2d49-4de7-94c4-853f7806f01a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:54.528201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a414ee28-2d49-4de7-94c4-853f7806f01a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9898df110866463ab0c30c61d9d872f7', 'request_id': 'req-a414ee28-2d49-4de7-94c4-853f7806f01a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:54.538324 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a414ee28-2d49-4de7-94c4-853f7806f01a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9898df110866463ab0c30c61d9d872f7', 'request_id': 'req-a414ee28-2d49-4de7-94c4-853f7806f01a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:54.541152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:54.547079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a414ee28-2d49-4de7-94c4-853f7806f01a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9898df110866463ab0c30c61d9d872f7', 'request_id': 'req-a414ee28-2d49-4de7-94c4-853f7806f01a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:54.565132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a414ee28-2d49-4de7-94c4-853f7806f01a admin admin] The token cache is being invalidate because project 9898df110866463ab0c30c61d9d872f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:04:54.565422 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a414ee28-2d49-4de7-94c4-853f7806f01a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:54.566058 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a414ee28-2d49-4de7-94c4-853f7806f01a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:54.567714 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1207/2404] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:04:54 2022] DELETE /identity/v3/projects/9898df110866463ab0c30c61d9d872f7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:54.572853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77fd50-f1c0-473a-a9f5-350b1bda4ff6 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:54.572853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77fd50-f1c0-473a-a9f5-350b1bda4ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:54.573551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77fd50-f1c0-473a-a9f5-350b1bda4ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:54.597756 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-8d77fd50-f1c0-473a-a9f5-350b1bda4ff6 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:54.600223 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-8d77fd50-f1c0-473a-a9f5-350b1bda4ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:54.665318 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1198/2405] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:54.674387 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea08c6b-d125-423d-bc1c-f6f55870c85e None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:54.674665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea08c6b-d125-423d-bc1c-f6f55870c85e None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:54.674959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea08c6b-d125-423d-bc1c-f6f55870c85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:54.695752 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77fd50-f1c0-473a-a9f5-350b1bda4ff6 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:54.697359 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-8d77fd50-f1c0-473a-a9f5-350b1bda4ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:54.703429 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-6ea08c6b-d125-423d-bc1c-f6f55870c85e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:54.704713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-6ea08c6b-d125-423d-bc1c-f6f55870c85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:54.714899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1208/2406] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:54.719281 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea08c6b-d125-423d-bc1c-f6f55870c85e None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:54.722954 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:54.728917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea08c6b-d125-423d-bc1c-f6f55870c85e None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:54.741045 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ab722-4fc5-4e65-869d-cb0470ff5523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:54.741667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ab722-4fc5-4e65-869d-cb0470ff5523 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:54.741969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ab722-4fc5-4e65-869d-cb0470ff5523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:54.742357 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ab722-4fc5-4e65-869d-cb0470ff5523 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:54.742939 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ab722-4fc5-4e65-869d-cb0470ff5523 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:04:54.743240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ab722-4fc5-4e65-869d-cb0470ff5523 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:54.744935 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ab722-4fc5-4e65-869d-cb0470ff5523 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:54.759956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1209/2407] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 15:04:54.767650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ab722-4fc5-4e65-869d-cb0470ff5523 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:54.783728 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ab722-4fc5-4e65-869d-cb0470ff5523 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:54.788533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453ff46-394e-4e68-97ad-7b84bafaa862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:54.789300 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453ff46-394e-4e68-97ad-7b84bafaa862 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:54.789646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453ff46-394e-4e68-97ad-7b84bafaa862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:54.789937 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453ff46-394e-4e68-97ad-7b84bafaa862 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:54.790625 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453ff46-394e-4e68-97ad-7b84bafaa862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:54.792585 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453ff46-394e-4e68-97ad-7b84bafaa862 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:54.796292 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77fd50-f1c0-473a-a9f5-350b1bda4ff6 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:54.801734 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc778b11-7bb3-4410-bb1e-a7e73eb1a775 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b6b34b4ee5492fb1f0699716e5118b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:54.802380 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc778b11-7bb3-4410-bb1e-a7e73eb1a775 service placement] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:54.802724 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc778b11-7bb3-4410-bb1e-a7e73eb1a775 service placement] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:54.803024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc778b11-7bb3-4410-bb1e-a7e73eb1a775 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:54.804772 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-cc778b11-7bb3-4410-bb1e-a7e73eb1a775 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:54.817963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc778b11-7bb3-4410-bb1e-a7e73eb1a775 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:54.819679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc778b11-7bb3-4410-bb1e-a7e73eb1a775 service placement] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:54.824653 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1210/2408] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:04:54 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:04:54.824653 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453ff46-394e-4e68-97ad-7b84bafaa862 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:54.824653 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2453ff46-394e-4e68-97ad-7b84bafaa862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:54.831109 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1199/2409] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:04:54.831915 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc778b11-7bb3-4410-bb1e-a7e73eb1a775 service placement] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:54.842057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc778b11-7bb3-4410-bb1e-a7e73eb1a775 service placement] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:54.893173 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453ff46-394e-4e68-97ad-7b84bafaa862 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:54.898044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805f5d3-431a-4c2a-a4bd-4c68d918b668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:54.898699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805f5d3-431a-4c2a-a4bd-4c68d918b668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:54.899108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805f5d3-431a-4c2a-a4bd-4c68d918b668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:54.899390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805f5d3-431a-4c2a-a4bd-4c68d918b668 admin admin] PATH_INFO: `/v3/users/f298f0c6a9e44efba2a1e2fc648a0f5d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:54.900563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805f5d3-431a-4c2a-a4bd-4c68d918b668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f298f0c6a9e44efba2a1e2fc648a0f5d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:54.902657 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805f5d3-431a-4c2a-a4bd-4c68d918b668 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:54.905900 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59d7da-0632-430a-8358-331f86e9cd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:54.906510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59d7da-0632-430a-8358-331f86e9cd7e admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:54.906821 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59d7da-0632-430a-8358-331f86e9cd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:54.907097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59d7da-0632-430a-8358-331f86e9cd7e admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:54.907735 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59d7da-0632-430a-8358-331f86e9cd7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:54.909643 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59d7da-0632-430a-8358-331f86e9cd7e admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:54.930989 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1200/2410] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:04:54 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:04:54.935991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59d7da-0632-430a-8358-331f86e9cd7e admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:54.945875 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59d7da-0632-430a-8358-331f86e9cd7e admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:54.951379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e31c0-5a96-4e76-8a6e-3e1e9b70ebbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:54.951687 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c805f5d3-431a-4c2a-a4bd-4c68d918b668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f298f0c6a9e44efba2a1e2fc648a0f5d'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:54.951993 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e31c0-5a96-4e76-8a6e-3e1e9b70ebbd admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:54.952269 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e31c0-5a96-4e76-8a6e-3e1e9b70ebbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:54.952541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e31c0-5a96-4e76-8a6e-3e1e9b70ebbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:54.953187 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e31c0-5a96-4e76-8a6e-3e1e9b70ebbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:54.954807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e31c0-5a96-4e76-8a6e-3e1e9b70ebbd admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:54.959991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c805f5d3-431a-4c2a-a4bd-4c68d918b668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f298f0c6a9e44efba2a1e2fc648a0f5d'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:54.965841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1201/2411] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:04:54.967308 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c805f5d3-431a-4c2a-a4bd-4c68d918b668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f298f0c6a9e44efba2a1e2fc648a0f5d'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:54.969782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e31c0-5a96-4e76-8a6e-3e1e9b70ebbd admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:54.979234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e31c0-5a96-4e76-8a6e-3e1e9b70ebbd admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:54.979377 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c805f5d3-431a-4c2a-a4bd-4c68d918b668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f298f0c6a9e44efba2a1e2fc648a0f5d'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:54.979941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-c805f5d3-431a-4c2a-a4bd-4c68d918b668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f298f0c6a9e44efba2a1e2fc648a0f5d'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:54.984658 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3f7d1-b07c-43e6-a373-f97f1addf4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:54.985249 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3f7d1-b07c-43e6-a373-f97f1addf4da admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:54.986276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1211/2412] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:04:54 2022] DELETE /identity/v3/users/f298f0c6a9e44efba2a1e2fc648a0f5d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:54.986417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3f7d1-b07c-43e6-a373-f97f1addf4da admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:54.986592 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3f7d1-b07c-43e6-a373-f97f1addf4da admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:54.987278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3f7d1-b07c-43e6-a373-f97f1addf4da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:54.989076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3f7d1-b07c-43e6-a373-f97f1addf4da admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:55.001539 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1202/2413] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:04:55.006013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805f5d3-431a-4c2a-a4bd-4c68d918b668 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:55.007749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c805f5d3-431a-4c2a-a4bd-4c68d918b668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:55.035160 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:55.085688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805f5d3-431a-4c2a-a4bd-4c68d918b668 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:55.091694 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bec10a-4d4f-4c4c-b40d-5b05c04f375f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:55.092428 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bec10a-4d4f-4c4c-b40d-5b05c04f375f admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:55.092857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bec10a-4d4f-4c4c-b40d-5b05c04f375f admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:55.093133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bec10a-4d4f-4c4c-b40d-5b05c04f375f admin admin] PATH_INFO: `/v3/projects/3a021020e4254a54b5892a11537a4807/users/59c1a8f322724f6490fe244c2704e1dd/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:55.108848 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bec10a-4d4f-4c4c-b40d-5b05c04f375f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a021020e4254a54b5892a11537a4807, user_id=59c1a8f322724f6490fe244c2704e1dd, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:55.111749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bec10a-4d4f-4c4c-b40d-5b05c04f375f admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:55.133298 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1212/2414] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:04:55 2022] PUT /identity/v3/projects/3a021020e4254a54b5892a11537a4807/users/59c1a8f322724f6490fe244c2704e1dd/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:55.138210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806fcce-5876-4a2d-aac0-257b0ff3d420 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:55.138498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806fcce-5876-4a2d-aac0-257b0ff3d420 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:55.138812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806fcce-5876-4a2d-aac0-257b0ff3d420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:55.158914 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-7806fcce-5876-4a2d-aac0-257b0ff3d420 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:55.160425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7806fcce-5876-4a2d-aac0-257b0ff3d420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:55.179408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bec10a-4d4f-4c4c-b40d-5b05c04f375f admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:55.181166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-66bec10a-4d4f-4c4c-b40d-5b05c04f375f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:55.232999 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1203/2415] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:55.243051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806fcce-5876-4a2d-aac0-257b0ff3d420 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:55.244399 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7806fcce-5876-4a2d-aac0-257b0ff3d420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:55.251136 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bec10a-4d4f-4c4c-b40d-5b05c04f375f admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:55.256874 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb9cc3-79d2-4ad1-9511-1fcdc7840154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:55.257514 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb9cc3-79d2-4ad1-9511-1fcdc7840154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:55.257794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb9cc3-79d2-4ad1-9511-1fcdc7840154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:55.258067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb9cc3-79d2-4ad1-9511-1fcdc7840154 admin admin] PATH_INFO: `/v3/projects/dc8a4487033d4e4aa231cc5873def0cf` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:55.259224 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb9cc3-79d2-4ad1-9511-1fcdc7840154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc8a4487033d4e4aa231cc5873def0cf)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:55.259807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:55.261200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb9cc3-79d2-4ad1-9511-1fcdc7840154 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:55.308597 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806fcce-5876-4a2d-aac0-257b0ff3d420 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:55.316405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f3fb9cc3-79d2-4ad1-9511-1fcdc7840154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc8a4487033d4e4aa231cc5873def0cf', 'request_id': 'req-f3fb9cc3-79d2-4ad1-9511-1fcdc7840154'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:55.333745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f3fb9cc3-79d2-4ad1-9511-1fcdc7840154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc8a4487033d4e4aa231cc5873def0cf', 'request_id': 'req-f3fb9cc3-79d2-4ad1-9511-1fcdc7840154'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:55.340631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f3fb9cc3-79d2-4ad1-9511-1fcdc7840154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc8a4487033d4e4aa231cc5873def0cf', 'request_id': 'req-f3fb9cc3-79d2-4ad1-9511-1fcdc7840154'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:55.358268 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f3fb9cc3-79d2-4ad1-9511-1fcdc7840154 admin admin] The token cache is being invalidate because project dc8a4487033d4e4aa231cc5873def0cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:04:55.358561 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f3fb9cc3-79d2-4ad1-9511-1fcdc7840154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:55.359267 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f3fb9cc3-79d2-4ad1-9511-1fcdc7840154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:55.360811 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1213/2416] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:04:55 2022] DELETE /identity/v3/projects/dc8a4487033d4e4aa231cc5873def0cf => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:55.366021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8f1a8-f414-48cd-9389-be753e226d03 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:55.366309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8f1a8-f414-48cd-9389-be753e226d03 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:55.366584 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8f1a8-f414-48cd-9389-be753e226d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:55.379807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4f9bb-6ce4-4f93-98b3-e8e6e84d42e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:55.380732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4f9bb-6ce4-4f93-98b3-e8e6e84d42e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:55.381216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4f9bb-6ce4-4f93-98b3-e8e6e84d42e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:55.381685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4f9bb-6ce4-4f93-98b3-e8e6e84d42e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:55.383686 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a9e4f9bb-6ce4-4f93-98b3-e8e6e84d42e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:55.388921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-4fb8f1a8-f414-48cd-9389-be753e226d03 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:55.390135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4fb8f1a8-f414-48cd-9389-be753e226d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:55.425342 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:55.461576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4f9bb-6ce4-4f93-98b3-e8e6e84d42e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:55.463059 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4f9bb-6ce4-4f93-98b3-e8e6e84d42e6 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:55.463203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1214/2417] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:04:55.471613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af3321-b044-4035-bb8f-d58850972740 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:55.471921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af3321-b044-4035-bb8f-d58850972740 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:55.472236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af3321-b044-4035-bb8f-d58850972740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:55.477650 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1204/2418] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:04:55.492853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-09af3321-b044-4035-bb8f-d58850972740 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:55.494083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-09af3321-b044-4035-bb8f-d58850972740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:55.502431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1215/2419] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:04:55.507405 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4f9bb-6ce4-4f93-98b3-e8e6e84d42e6 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:55.517132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4f9bb-6ce4-4f93-98b3-e8e6e84d42e6 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:55.521906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706aa92-706f-4623-947d-e00abedcf784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:55.522519 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706aa92-706f-4623-947d-e00abedcf784 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:55.522828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706aa92-706f-4623-947d-e00abedcf784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:55.523174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706aa92-706f-4623-947d-e00abedcf784 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:55.523811 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706aa92-706f-4623-947d-e00abedcf784 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:04:55.524112 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706aa92-706f-4623-947d-e00abedcf784 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:55.525714 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706aa92-706f-4623-947d-e00abedcf784 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:55.538624 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1205/2420] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:04:55.543225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af3321-b044-4035-bb8f-d58850972740 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:55.553686 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af3321-b044-4035-bb8f-d58850972740 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:55.558933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b097c2-13e6-4399-9ff9-013a7e5c92f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:55.559530 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b097c2-13e6-4399-9ff9-013a7e5c92f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:55.559866 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b097c2-13e6-4399-9ff9-013a7e5c92f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:55.560244 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b097c2-13e6-4399-9ff9-013a7e5c92f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:55.560895 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b097c2-13e6-4399-9ff9-013a7e5c92f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:55.562840 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b097c2-13e6-4399-9ff9-013a7e5c92f5 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:55.592648 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1216/2421] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:04:55 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:04:55.596613 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706aa92-706f-4623-947d-e00abedcf784 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:55.607605 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706aa92-706f-4623-947d-e00abedcf784 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:55.666851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0775d2-66f9-4dbd-9403-e18997e89483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:55.667475 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0775d2-66f9-4dbd-9403-e18997e89483 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:55.667755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0775d2-66f9-4dbd-9403-e18997e89483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:55.668029 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0775d2-66f9-4dbd-9403-e18997e89483 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:55.668716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0775d2-66f9-4dbd-9403-e18997e89483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:55.671420 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0775d2-66f9-4dbd-9403-e18997e89483 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:55.683509 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:55.695878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1206/2422] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:04:55 2022] POST /identity/v3/users => generated 303 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:04:55.700154 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b097c2-13e6-4399-9ff9-013a7e5c92f5 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:55.711406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b097c2-13e6-4399-9ff9-013a7e5c92f5 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:55.716877 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa545c-13ea-4a6a-8db4-9150fea89489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:55.717491 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa545c-13ea-4a6a-8db4-9150fea89489 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:55.717770 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa545c-13ea-4a6a-8db4-9150fea89489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:55.718042 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa545c-13ea-4a6a-8db4-9150fea89489 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:55.718691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa545c-13ea-4a6a-8db4-9150fea89489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:55.720310 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa545c-13ea-4a6a-8db4-9150fea89489 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:55.731262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1217/2423] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:04:55.735603 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0775d2-66f9-4dbd-9403-e18997e89483 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:55.746453 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0775d2-66f9-4dbd-9403-e18997e89483 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:55.752077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760c9ee-bf44-442f-b7ba-a1ce89d960ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:55.752796 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760c9ee-bf44-442f-b7ba-a1ce89d960ce admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:55.753100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760c9ee-bf44-442f-b7ba-a1ce89d960ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:55.753377 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760c9ee-bf44-442f-b7ba-a1ce89d960ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:55.754027 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760c9ee-bf44-442f-b7ba-a1ce89d960ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:55.755686 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760c9ee-bf44-442f-b7ba-a1ce89d960ce admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:55.767292 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1207/2424] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:04:55.771526 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa545c-13ea-4a6a-8db4-9150fea89489 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:55.783789 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa545c-13ea-4a6a-8db4-9150fea89489 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:55.789502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee76f10-c4f1-46b8-8155-c4e292eb50c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:55.790310 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee76f10-c4f1-46b8-8155-c4e292eb50c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:55.790660 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee76f10-c4f1-46b8-8155-c4e292eb50c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:55.791099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee76f10-c4f1-46b8-8155-c4e292eb50c4 admin admin] PATH_INFO: `/v3/projects/b6e8f8d3d000475d81d19b0c98d37057/users/126f155956e04e959ddabfc9c19b6031/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:55.815573 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee76f10-c4f1-46b8-8155-c4e292eb50c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6e8f8d3d000475d81d19b0c98d37057, user_id=126f155956e04e959ddabfc9c19b6031, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:55.818394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee76f10-c4f1-46b8-8155-c4e292eb50c4 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:55.834248 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1218/2425] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:04:55 2022] PUT /identity/v3/projects/b6e8f8d3d000475d81d19b0c98d37057/users/126f155956e04e959ddabfc9c19b6031/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:55.845135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899feae-abea-4cdd-b598-7aa004532862 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:55.845338 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899feae-abea-4cdd-b598-7aa004532862 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:55.845594 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899feae-abea-4cdd-b598-7aa004532862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:55.880688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-b899feae-abea-4cdd-b598-7aa004532862 None None] MFA Rules not processed for user `126f155956e04e959ddabfc9c19b6031`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:55.881057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b899feae-abea-4cdd-b598-7aa004532862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:55.956206 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1208/2426] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:55 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:55.960811 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee76f10-c4f1-46b8-8155-c4e292eb50c4 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:56.025130 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee76f10-c4f1-46b8-8155-c4e292eb50c4 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:56.030601 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06a08f-9e21-4b1e-96b5-d799393e1310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:56.031423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06a08f-9e21-4b1e-96b5-d799393e1310 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:56.031588 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06a08f-9e21-4b1e-96b5-d799393e1310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:56.031769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06a08f-9e21-4b1e-96b5-d799393e1310 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:56.032423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06a08f-9e21-4b1e-96b5-d799393e1310 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:56.034349 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06a08f-9e21-4b1e-96b5-d799393e1310 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:56.067510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1219/2427] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:04:55 2022] POST /identity/v3/projects => generated 405 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:04:56.071932 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899feae-abea-4cdd-b598-7aa004532862 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:56.092956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899feae-abea-4cdd-b598-7aa004532862 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:56.176522 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87753288-1737-42a5-bc5f-1e14b7999e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:56.177345 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87753288-1737-42a5-bc5f-1e14b7999e17 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:56.177753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87753288-1737-42a5-bc5f-1e14b7999e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:56.179185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87753288-1737-42a5-bc5f-1e14b7999e17 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:56.179185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87753288-1737-42a5-bc5f-1e14b7999e17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:56.180959 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87753288-1737-42a5-bc5f-1e14b7999e17 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:56.208407 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1209/2428] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:04:56 2022] POST /identity/v3/users => generated 309 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:04:56.212550 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06a08f-9e21-4b1e-96b5-d799393e1310 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:56.224556 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06a08f-9e21-4b1e-96b5-d799393e1310 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:56.230219 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590379fd-cf1e-4e6e-9117-676d3327d692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:56.230911 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590379fd-cf1e-4e6e-9117-676d3327d692 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:56.231193 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590379fd-cf1e-4e6e-9117-676d3327d692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:56.231466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590379fd-cf1e-4e6e-9117-676d3327d692 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:56.232158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590379fd-cf1e-4e6e-9117-676d3327d692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:56.233785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590379fd-cf1e-4e6e-9117-676d3327d692 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:56.245774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1220/2429] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:04:56.250598 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87753288-1737-42a5-bc5f-1e14b7999e17 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:56.261498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87753288-1737-42a5-bc5f-1e14b7999e17 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:56.267099 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177cd1e-da8b-4824-ba0a-c3cc13219f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:56.267783 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177cd1e-da8b-4824-ba0a-c3cc13219f13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:56.268067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177cd1e-da8b-4824-ba0a-c3cc13219f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:56.268344 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177cd1e-da8b-4824-ba0a-c3cc13219f13 admin admin] PATH_INFO: `/v3/projects/18dea18a9b6c426d8c6411b67654446e/users/21c7b59955c141eda68c6a5a4af5d0f7/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:56.285418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8177cd1e-da8b-4824-ba0a-c3cc13219f13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18dea18a9b6c426d8c6411b67654446e, user_id=21c7b59955c141eda68c6a5a4af5d0f7, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:56.288355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8177cd1e-da8b-4824-ba0a-c3cc13219f13 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:56.307332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1210/2430] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:04:56 2022] PUT /identity/v3/projects/18dea18a9b6c426d8c6411b67654446e/users/21c7b59955c141eda68c6a5a4af5d0f7/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:56.308471 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590379fd-cf1e-4e6e-9117-676d3327d692 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:56.319351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590379fd-cf1e-4e6e-9117-676d3327d692 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:56.370881 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:56.387379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234877a3-eb68-47f7-baa9-190f9dbcf2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:56.388767 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234877a3-eb68-47f7-baa9-190f9dbcf2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:56.389369 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234877a3-eb68-47f7-baa9-190f9dbcf2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:56.389739 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234877a3-eb68-47f7-baa9-190f9dbcf2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:56.390831 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234877a3-eb68-47f7-baa9-190f9dbcf2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:56.392533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234877a3-eb68-47f7-baa9-190f9dbcf2d9 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:56.407746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1221/2431] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:04:56.408996 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177cd1e-da8b-4824-ba0a-c3cc13219f13 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:56.420574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177cd1e-da8b-4824-ba0a-c3cc13219f13 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:56.425817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de23a5-5705-4197-b1ec-15cbbdb7ec74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:56.426485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de23a5-5705-4197-b1ec-15cbbdb7ec74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:56.426792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de23a5-5705-4197-b1ec-15cbbdb7ec74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:56.427072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de23a5-5705-4197-b1ec-15cbbdb7ec74 admin admin] PATH_INFO: `/v3/projects/18dea18a9b6c426d8c6411b67654446e/users/21c7b59955c141eda68c6a5a4af5d0f7/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:56.428999 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de23a5-5705-4197-b1ec-15cbbdb7ec74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18dea18a9b6c426d8c6411b67654446e, user_id=21c7b59955c141eda68c6a5a4af5d0f7, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:56.431693 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de23a5-5705-4197-b1ec-15cbbdb7ec74 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:56.445349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1211/2432] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:04:56 2022] PUT /identity/v3/projects/18dea18a9b6c426d8c6411b67654446e/users/21c7b59955c141eda68c6a5a4af5d0f7/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:56.458560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698d246-c371-404f-b4e5-0c07312b6788 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:56.458994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698d246-c371-404f-b4e5-0c07312b6788 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:56.459291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698d246-c371-404f-b4e5-0c07312b6788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:56.492701 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-e698d246-c371-404f-b4e5-0c07312b6788 None None] MFA Rules not processed for user `21c7b59955c141eda68c6a5a4af5d0f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:56.494038 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e698d246-c371-404f-b4e5-0c07312b6788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:56.613073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1222/2433] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:56 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:56.622166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de23a5-5705-4197-b1ec-15cbbdb7ec74 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:56.625299 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e1de23a5-5705-4197-b1ec-15cbbdb7ec74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:56.659778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:56.697440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de23a5-5705-4197-b1ec-15cbbdb7ec74 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:56.702072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7bb48-bb6d-4fde-b8a6-378b7d0f4866 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:56.702838 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7bb48-bb6d-4fde-b8a6-378b7d0f4866 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:56.703261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7bb48-bb6d-4fde-b8a6-378b7d0f4866 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:56.703613 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7bb48-bb6d-4fde-b8a6-378b7d0f4866 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:56.714470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7bb48-bb6d-4fde-b8a6-378b7d0f4866 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:56.716025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7bb48-bb6d-4fde-b8a6-378b7d0f4866 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:56.726943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1212/2434] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:56 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:04:56.883900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698d246-c371-404f-b4e5-0c07312b6788 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:56.960098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698d246-c371-404f-b4e5-0c07312b6788 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:56.966557 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739ad04-f475-4b62-9e7d-062071a03acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:56.967319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739ad04-f475-4b62-9e7d-062071a03acd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:56.967550 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739ad04-f475-4b62-9e7d-062071a03acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:56.968423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739ad04-f475-4b62-9e7d-062071a03acd admin admin] PATH_INFO: `/v3/users/126f155956e04e959ddabfc9c19b6031` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:56.969169 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739ad04-f475-4b62-9e7d-062071a03acd admin admin] RBAC: Authorizing `identity:delete_user(user_id=126f155956e04e959ddabfc9c19b6031)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:56.972779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739ad04-f475-4b62-9e7d-062071a03acd admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:57.055856 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b739ad04-f475-4b62-9e7d-062071a03acd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '126f155956e04e959ddabfc9c19b6031'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:57.064281 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b739ad04-f475-4b62-9e7d-062071a03acd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '126f155956e04e959ddabfc9c19b6031'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:57.074340 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b739ad04-f475-4b62-9e7d-062071a03acd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '126f155956e04e959ddabfc9c19b6031'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:57.081760 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:57.092609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b739ad04-f475-4b62-9e7d-062071a03acd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '126f155956e04e959ddabfc9c19b6031'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:57.094243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b739ad04-f475-4b62-9e7d-062071a03acd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '126f155956e04e959ddabfc9c19b6031'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:57.103295 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1223/2435] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:04:56 2022] DELETE /identity/v3/users/126f155956e04e959ddabfc9c19b6031 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:57.108138 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d8ba4-4dc9-44ef-8034-ef38e9389b58 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:57.108575 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d8ba4-4dc9-44ef-8034-ef38e9389b58 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:57.108889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d8ba4-4dc9-44ef-8034-ef38e9389b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:57.128999 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-640d8ba4-4dc9-44ef-8034-ef38e9389b58 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:57.130716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-640d8ba4-4dc9-44ef-8034-ef38e9389b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:57.227148 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1213/2436] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:04:57.241328 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739ad04-f475-4b62-9e7d-062071a03acd admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:57.243377 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b739ad04-f475-4b62-9e7d-062071a03acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:57.321024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739ad04-f475-4b62-9e7d-062071a03acd admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:57.325968 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af70ba5-8860-4b35-a29d-458544321867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:57.326658 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af70ba5-8860-4b35-a29d-458544321867 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:57.327053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af70ba5-8860-4b35-a29d-458544321867 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:57.327406 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af70ba5-8860-4b35-a29d-458544321867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:57.346162 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af70ba5-8860-4b35-a29d-458544321867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:57.348687 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af70ba5-8860-4b35-a29d-458544321867 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:57.367789 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1224/2437] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:04:57.702294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d8ba4-4dc9-44ef-8034-ef38e9389b58 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:57.704479 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-640d8ba4-4dc9-44ef-8034-ef38e9389b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:57.719741 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d8ba4-4dc9-44ef-8034-ef38e9389b58 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:57.731528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f00ec-7b59-4bd9-a716-101861db45b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:57.733023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f00ec-7b59-4bd9-a716-101861db45b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:57.733369 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f00ec-7b59-4bd9-a716-101861db45b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:57.733730 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f00ec-7b59-4bd9-a716-101861db45b0 admin admin] PATH_INFO: `/v3/projects/b6e8f8d3d000475d81d19b0c98d37057` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:57.735168 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f00ec-7b59-4bd9-a716-101861db45b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6e8f8d3d000475d81d19b0c98d37057)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:57.737753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f00ec-7b59-4bd9-a716-101861db45b0 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:57.791870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0f2f00ec-7b59-4bd9-a716-101861db45b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6e8f8d3d000475d81d19b0c98d37057', 'request_id': 'req-0f2f00ec-7b59-4bd9-a716-101861db45b0'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:57.802965 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0f2f00ec-7b59-4bd9-a716-101861db45b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6e8f8d3d000475d81d19b0c98d37057', 'request_id': 'req-0f2f00ec-7b59-4bd9-a716-101861db45b0'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:57.810064 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0f2f00ec-7b59-4bd9-a716-101861db45b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6e8f8d3d000475d81d19b0c98d37057', 'request_id': 'req-0f2f00ec-7b59-4bd9-a716-101861db45b0'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:57.822388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0f2f00ec-7b59-4bd9-a716-101861db45b0 admin admin] The token cache is being invalidate because project b6e8f8d3d000475d81d19b0c98d37057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:04:57.822704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0f2f00ec-7b59-4bd9-a716-101861db45b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:57.823688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0f2f00ec-7b59-4bd9-a716-101861db45b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:57.825750 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1214/2438] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:04:57 2022] DELETE /identity/v3/projects/b6e8f8d3d000475d81d19b0c98d37057 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:57.829684 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af70ba5-8860-4b35-a29d-458544321867 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:57.840776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af70ba5-8860-4b35-a29d-458544321867 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:57.920102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74823e4b-0c5d-4a74-acaf-cf4decf71da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:57.920914 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74823e4b-0c5d-4a74-acaf-cf4decf71da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:57.921186 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74823e4b-0c5d-4a74-acaf-cf4decf71da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:57.921528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74823e4b-0c5d-4a74-acaf-cf4decf71da9 admin admin] PATH_INFO: `/v3/users/21c7b59955c141eda68c6a5a4af5d0f7` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:57.922786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74823e4b-0c5d-4a74-acaf-cf4decf71da9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21c7b59955c141eda68c6a5a4af5d0f7)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:57.924827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74823e4b-0c5d-4a74-acaf-cf4decf71da9 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:57.982784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-74823e4b-0c5d-4a74-acaf-cf4decf71da9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21c7b59955c141eda68c6a5a4af5d0f7'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:57.992463 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-74823e4b-0c5d-4a74-acaf-cf4decf71da9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21c7b59955c141eda68c6a5a4af5d0f7'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:58.000922 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-74823e4b-0c5d-4a74-acaf-cf4decf71da9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21c7b59955c141eda68c6a5a4af5d0f7'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:58.014619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-74823e4b-0c5d-4a74-acaf-cf4decf71da9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21c7b59955c141eda68c6a5a4af5d0f7'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:58.015391 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-74823e4b-0c5d-4a74-acaf-cf4decf71da9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21c7b59955c141eda68c6a5a4af5d0f7'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:58.023255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1225/2439] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:04:57 2022] DELETE /identity/v3/users/21c7b59955c141eda68c6a5a4af5d0f7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:04:58.382175 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f00ec-7b59-4bd9-a716-101861db45b0 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:58.383952 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-0f2f00ec-7b59-4bd9-a716-101861db45b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:58.416733 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f00ec-7b59-4bd9-a716-101861db45b0 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:58.483760 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:58.503802 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28cd3ff-194b-4069-a435-19eec3b3fa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:58.504463 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cd3ff-194b-4069-a435-19eec3b3fa04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:58.504743 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cd3ff-194b-4069-a435-19eec3b3fa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:58.505044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cd3ff-194b-4069-a435-19eec3b3fa04 admin admin] PATH_INFO: `/v3/projects/18dea18a9b6c426d8c6411b67654446e` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:58.506158 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28cd3ff-194b-4069-a435-19eec3b3fa04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18dea18a9b6c426d8c6411b67654446e)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:58.508117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28cd3ff-194b-4069-a435-19eec3b3fa04 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:58.570725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c28cd3ff-194b-4069-a435-19eec3b3fa04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18dea18a9b6c426d8c6411b67654446e', 'request_id': 'req-c28cd3ff-194b-4069-a435-19eec3b3fa04'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:58.581729 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c28cd3ff-194b-4069-a435-19eec3b3fa04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18dea18a9b6c426d8c6411b67654446e', 'request_id': 'req-c28cd3ff-194b-4069-a435-19eec3b3fa04'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:58.593218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c28cd3ff-194b-4069-a435-19eec3b3fa04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18dea18a9b6c426d8c6411b67654446e', 'request_id': 'req-c28cd3ff-194b-4069-a435-19eec3b3fa04'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:58.612128 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c28cd3ff-194b-4069-a435-19eec3b3fa04 admin admin] The token cache is being invalidate because project 18dea18a9b6c426d8c6411b67654446e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:04:58.612481 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c28cd3ff-194b-4069-a435-19eec3b3fa04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:58.613115 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-c28cd3ff-194b-4069-a435-19eec3b3fa04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:04:58.615779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1215/2440] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:04:58 2022] DELETE /identity/v3/projects/18dea18a9b6c426d8c6411b67654446e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:04:58.620545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce8f96-2ee8-44a6-a7bc-f3839f6c4a39 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:58.623517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce8f96-2ee8-44a6-a7bc-f3839f6c4a39 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:58.629420 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce8f96-2ee8-44a6-a7bc-f3839f6c4a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:58.654336 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-8bce8f96-2ee8-44a6-a7bc-f3839f6c4a39 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:58.655279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-8bce8f96-2ee8-44a6-a7bc-f3839f6c4a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:58.662706 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:58.765754 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1226/2441] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:58.774935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e4cb8-10b6-4986-abf7-d5421a702ef6 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:58.775437 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e4cb8-10b6-4986-abf7-d5421a702ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:58.775521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e4cb8-10b6-4986-abf7-d5421a702ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:58.799470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-a07e4cb8-10b6-4986-abf7-d5421a702ef6 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:58.801078 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a07e4cb8-10b6-4986-abf7-d5421a702ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:58.809837 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1216/2442] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:58.814875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce8f96-2ee8-44a6-a7bc-f3839f6c4a39 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:58.825266 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce8f96-2ee8-44a6-a7bc-f3839f6c4a39 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:58.831658 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c58a5-27dd-4eec-a8a4-62f1068db7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:58.832219 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c58a5-27dd-4eec-a8a4-62f1068db7d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:58.832672 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c58a5-27dd-4eec-a8a4-62f1068db7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:58.833037 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c58a5-27dd-4eec-a8a4-62f1068db7d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:58.833751 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c58a5-27dd-4eec-a8a4-62f1068db7d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:04:58.834060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c58a5-27dd-4eec-a8a4-62f1068db7d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:58.836724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c58a5-27dd-4eec-a8a4-62f1068db7d3 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:58.850857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1227/2443] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:04:58.855737 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e4cb8-10b6-4986-abf7-d5421a702ef6 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:58.868860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e4cb8-10b6-4986-abf7-d5421a702ef6 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:58.876362 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e650d-4ec3-415a-aca8-2cab9d478c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:58.877118 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e650d-4ec3-415a-aca8-2cab9d478c32 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:58.877518 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e650d-4ec3-415a-aca8-2cab9d478c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:58.877885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e650d-4ec3-415a-aca8-2cab9d478c32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:58.879095 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e650d-4ec3-415a-aca8-2cab9d478c32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:58.882406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e650d-4ec3-415a-aca8-2cab9d478c32 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:58.924834 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1217/2444] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:04:58 2022] POST /identity/v3/projects => generated 389 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:04:58.926012 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c58a5-27dd-4eec-a8a4-62f1068db7d3 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:58.937501 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c58a5-27dd-4eec-a8a4-62f1068db7d3 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:59.026759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016e802-581a-42ae-baf5-f67c056cc56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:59.031428 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016e802-581a-42ae-baf5-f67c056cc56c admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:59.031711 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016e802-581a-42ae-baf5-f67c056cc56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:59.031984 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016e802-581a-42ae-baf5-f67c056cc56c admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:59.032653 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016e802-581a-42ae-baf5-f67c056cc56c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:59.034604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016e802-581a-42ae-baf5-f67c056cc56c admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:59.070386 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1228/2445] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:04:58 2022] POST /identity/v3/users => generated 295 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:04:59.071862 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e650d-4ec3-415a-aca8-2cab9d478c32 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:59.091983 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e650d-4ec3-415a-aca8-2cab9d478c32 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:59.100512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed0a3c-509f-4fb0-aaf2-8d1a8e42b1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:59.102489 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed0a3c-509f-4fb0-aaf2-8d1a8e42b1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:59.103890 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed0a3c-509f-4fb0-aaf2-8d1a8e42b1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:59.104983 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed0a3c-509f-4fb0-aaf2-8d1a8e42b1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:59.107430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed0a3c-509f-4fb0-aaf2-8d1a8e42b1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:59.110116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ed0a3c-509f-4fb0-aaf2-8d1a8e42b1b8 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:59.126797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1218/2446] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:04:59.129825 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016e802-581a-42ae-baf5-f67c056cc56c admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:59.141238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016e802-581a-42ae-baf5-f67c056cc56c admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:59.147299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d90908-b4af-4c89-9480-21e35018281d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:59.148078 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d90908-b4af-4c89-9480-21e35018281d admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:59.148458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d90908-b4af-4c89-9480-21e35018281d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:59.148829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d90908-b4af-4c89-9480-21e35018281d admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:59.149760 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d90908-b4af-4c89-9480-21e35018281d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:59.151799 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d90908-b4af-4c89-9480-21e35018281d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:59.174238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1229/2447] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:04:59.178521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed0a3c-509f-4fb0-aaf2-8d1a8e42b1b8 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:59.202421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ed0a3c-509f-4fb0-aaf2-8d1a8e42b1b8 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:59.212576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99f740-7df0-4038-9920-de5754b97f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:59.213407 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99f740-7df0-4038-9920-de5754b97f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:59.213774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99f740-7df0-4038-9920-de5754b97f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:59.214169 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99f740-7df0-4038-9920-de5754b97f12 admin admin] PATH_INFO: `/v3/projects/465b27e83c83460393d42cd40d8db34a/users/7342d4cfe9e24b00a58a359ff320215f/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:59.235489 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99f740-7df0-4038-9920-de5754b97f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=465b27e83c83460393d42cd40d8db34a, user_id=7342d4cfe9e24b00a58a359ff320215f, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:59.238637 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99f740-7df0-4038-9920-de5754b97f12 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:59.267807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1219/2448] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:04:59 2022] PUT /identity/v3/projects/465b27e83c83460393d42cd40d8db34a/users/7342d4cfe9e24b00a58a359ff320215f/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:04:59.269865 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae3446-cd3d-410c-84b6-d75bec73ba11 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:59.270298 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae3446-cd3d-410c-84b6-d75bec73ba11 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:59.271380 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae3446-cd3d-410c-84b6-d75bec73ba11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:59.333869 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-50ae3446-cd3d-410c-84b6-d75bec73ba11 None None] MFA Rules not processed for user `7342d4cfe9e24b00a58a359ff320215f`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:04:59.335672 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-50ae3446-cd3d-410c-84b6-d75bec73ba11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:59.421907 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1230/2449] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:04:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:04:59.426167 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99f740-7df0-4038-9920-de5754b97f12 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:59.483837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae3446-cd3d-410c-84b6-d75bec73ba11 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:59.485410 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-50ae3446-cd3d-410c-84b6-d75bec73ba11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:59.493127 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:59.537839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99f740-7df0-4038-9920-de5754b97f12 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:59.551434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17a3b2-51ca-4f25-9370-66f6daea8ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:59.552080 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17a3b2-51ca-4f25-9370-66f6daea8ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:59.552450 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17a3b2-51ca-4f25-9370-66f6daea8ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:59.552921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17a3b2-51ca-4f25-9370-66f6daea8ee0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:59.561952 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17a3b2-51ca-4f25-9370-66f6daea8ee0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:59.569870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17a3b2-51ca-4f25-9370-66f6daea8ee0 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:59.606713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae3446-cd3d-410c-84b6-d75bec73ba11 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:59.619119 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350318d-2e46-4272-a9c1-98747050ee48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:59.620076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350318d-2e46-4272-a9c1-98747050ee48 service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:59.620467 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350318d-2e46-4272-a9c1-98747050ee48 service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:59.620842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350318d-2e46-4272-a9c1-98747050ee48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:59.623305 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d350318d-2e46-4272-a9c1-98747050ee48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:04:59.625474 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1220/2450] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:04:59 2022] POST /identity/v3/projects => generated 389 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:04:59.630077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17a3b2-51ca-4f25-9370-66f6daea8ee0 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:59.637648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:04:59.701506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350318d-2e46-4272-a9c1-98747050ee48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:59.704357 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350318d-2e46-4272-a9c1-98747050ee48 service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:59.718892 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1231/2451] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:04:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:04:59.722977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17a3b2-51ca-4f25-9370-66f6daea8ee0 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:59.728166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73619e5-befc-49c1-8eea-0e7be4a7f003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:59.728842 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73619e5-befc-49c1-8eea-0e7be4a7f003 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:59.729168 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73619e5-befc-49c1-8eea-0e7be4a7f003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:59.729445 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73619e5-befc-49c1-8eea-0e7be4a7f003 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:59.730634 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73619e5-befc-49c1-8eea-0e7be4a7f003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:59.732289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73619e5-befc-49c1-8eea-0e7be4a7f003 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:59.814356 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1221/2452] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:04:59 2022] POST /identity/v3/users => generated 301 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:04:59.818781 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350318d-2e46-4272-a9c1-98747050ee48 service nova] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:59.849695 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350318d-2e46-4272-a9c1-98747050ee48 service nova] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:59.878282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d35c5b-c245-4b00-b11f-5a6991e11aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:59.878965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d35c5b-c245-4b00-b11f-5a6991e11aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:59.879244 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d35c5b-c245-4b00-b11f-5a6991e11aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:59.879511 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d35c5b-c245-4b00-b11f-5a6991e11aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:59.880351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d35c5b-c245-4b00-b11f-5a6991e11aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:59.882039 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d35c5b-c245-4b00-b11f-5a6991e11aa1 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:59.903786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1232/2453] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:04:59.904288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73619e5-befc-49c1-8eea-0e7be4a7f003 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:04:59.919185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73619e5-befc-49c1-8eea-0e7be4a7f003 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:04:59.929033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1853881-f313-446b-823e-26588e09be34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:04:59.930233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1853881-f313-446b-823e-26588e09be34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:04:59.930991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1853881-f313-446b-823e-26588e09be34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:04:59.933059 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1853881-f313-446b-823e-26588e09be34 admin admin] PATH_INFO: `/v3/projects/4dff1e9ac01d427cbc579365ce7ccd4e/users/d3ba571dd5a34abca24d47fb2b466aa0/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:04:59.957970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1853881-f313-446b-823e-26588e09be34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dff1e9ac01d427cbc579365ce7ccd4e, user_id=d3ba571dd5a34abca24d47fb2b466aa0, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:04:59.966313 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1853881-f313-446b-823e-26588e09be34 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:04:59.996055 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1222/2454] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:04:59 2022] PUT /identity/v3/projects/4dff1e9ac01d427cbc579365ce7ccd4e/users/d3ba571dd5a34abca24d47fb2b466aa0/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:04:59.999880 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d35c5b-c245-4b00-b11f-5a6991e11aa1 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:00.012226 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d35c5b-c245-4b00-b11f-5a6991e11aa1 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:00.070383 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae59a02-ee56-4dbd-8303-c8e15dcbb9d8 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:00.071020 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae59a02-ee56-4dbd-8303-c8e15dcbb9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:00.072039 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae59a02-ee56-4dbd-8303-c8e15dcbb9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:00.111884 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-aae59a02-ee56-4dbd-8303-c8e15dcbb9d8 None None] MFA Rules not processed for user `59c1a8f322724f6490fe244c2704e1dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:00.113398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-aae59a02-ee56-4dbd-8303-c8e15dcbb9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:00.115398 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc694d-b992-4be2-8137-6ec1ec6445d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:00.116025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc694d-b992-4be2-8137-6ec1ec6445d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:00.116345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc694d-b992-4be2-8137-6ec1ec6445d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:00.116557 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc694d-b992-4be2-8137-6ec1ec6445d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:00.117203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc694d-b992-4be2-8137-6ec1ec6445d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:00.118900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc694d-b992-4be2-8137-6ec1ec6445d6 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:00.131699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1233/2455] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:00.136092 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc694d-b992-4be2-8137-6ec1ec6445d6 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:00.177783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc694d-b992-4be2-8137-6ec1ec6445d6 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:00.185192 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90899f9e-ff6f-4528-b04b-9aea0ef1ec79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:00.185859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90899f9e-ff6f-4528-b04b-9aea0ef1ec79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:00.186130 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90899f9e-ff6f-4528-b04b-9aea0ef1ec79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:00.186575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90899f9e-ff6f-4528-b04b-9aea0ef1ec79 admin admin] PATH_INFO: `/v3/projects/4dff1e9ac01d427cbc579365ce7ccd4e/users/d3ba571dd5a34abca24d47fb2b466aa0/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:00.188968 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90899f9e-ff6f-4528-b04b-9aea0ef1ec79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dff1e9ac01d427cbc579365ce7ccd4e, user_id=d3ba571dd5a34abca24d47fb2b466aa0, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:00.191887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90899f9e-ff6f-4528-b04b-9aea0ef1ec79 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:00.202869 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1223/2456] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:00.204939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae59a02-ee56-4dbd-8303-c8e15dcbb9d8 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:00.206097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1234/2457] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:00 2022] PUT /identity/v3/projects/4dff1e9ac01d427cbc579365ce7ccd4e/users/d3ba571dd5a34abca24d47fb2b466aa0/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:00.206566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-aae59a02-ee56-4dbd-8303-c8e15dcbb9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:00.220741 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb40b70-db96-40bc-a562-b203c304a4d4 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:00.221008 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb40b70-db96-40bc-a562-b203c304a4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:00.221287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb40b70-db96-40bc-a562-b203c304a4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:00.228404 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae59a02-ee56-4dbd-8303-c8e15dcbb9d8 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:00.265369 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:00.271224 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-edb40b70-db96-40bc-a562-b203c304a4d4 None None] MFA Rules not processed for user `d3ba571dd5a34abca24d47fb2b466aa0`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:00.272651 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-edb40b70-db96-40bc-a562-b203c304a4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:00.316442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3a51a-ae79-4685-b41b-201f60a87853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:00.317890 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3a51a-ae79-4685-b41b-201f60a87853 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:00.318292 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3a51a-ae79-4685-b41b-201f60a87853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:00.318957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3a51a-ae79-4685-b41b-201f60a87853 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:00.320601 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff3a51a-ae79-4685-b41b-201f60a87853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:00.327501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff3a51a-ae79-4685-b41b-201f60a87853 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:00.337615 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:00.364135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1224/2458] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:05:00 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:05:00.370237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3a51a-ae79-4685-b41b-201f60a87853 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:00.380187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1235/2459] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:00 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:05:00.390249 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb40b70-db96-40bc-a562-b203c304a4d4 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:00.392889 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-edb40b70-db96-40bc-a562-b203c304a4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:00.451847 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3a51a-ae79-4685-b41b-201f60a87853 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:00.457399 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688c750-2b9b-4f0f-b0ea-3d58bed1a3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:00.458081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688c750-2b9b-4f0f-b0ea-3d58bed1a3d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:00.458503 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688c750-2b9b-4f0f-b0ea-3d58bed1a3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:00.458918 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7688c750-2b9b-4f0f-b0ea-3d58bed1a3d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:00.459881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688c750-2b9b-4f0f-b0ea-3d58bed1a3d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:00.461942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7688c750-2b9b-4f0f-b0ea-3d58bed1a3d0 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:00.470443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb40b70-db96-40bc-a562-b203c304a4d4 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:00.490067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1225/2460] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:05:00 2022] POST /identity/v3/users => generated 298 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:00.491173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688c750-2b9b-4f0f-b0ea-3d58bed1a3d0 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:00.502111 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7688c750-2b9b-4f0f-b0ea-3d58bed1a3d0 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:00.507628 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757f2e1-269e-4707-ae70-75c07f01d0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:00.508247 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757f2e1-269e-4707-ae70-75c07f01d0fe admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:00.508688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757f2e1-269e-4707-ae70-75c07f01d0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:00.508968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757f2e1-269e-4707-ae70-75c07f01d0fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:00.509625 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757f2e1-269e-4707-ae70-75c07f01d0fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:00.511963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757f2e1-269e-4707-ae70-75c07f01d0fe admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:00.524696 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1226/2461] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:00.529270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757f2e1-269e-4707-ae70-75c07f01d0fe admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:00.534021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf2624-59e9-4506-8d50-26c714bccf03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:00.534912 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf2624-59e9-4506-8d50-26c714bccf03 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:00.535184 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf2624-59e9-4506-8d50-26c714bccf03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:00.535453 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf2624-59e9-4506-8d50-26c714bccf03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:00.541250 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757f2e1-269e-4707-ae70-75c07f01d0fe admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:00.548562 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448daa7-f319-4abd-8a6a-e0c8102c4255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:00.549350 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448daa7-f319-4abd-8a6a-e0c8102c4255 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:00.549752 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448daa7-f319-4abd-8a6a-e0c8102c4255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:00.550103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448daa7-f319-4abd-8a6a-e0c8102c4255 admin admin] PATH_INFO: `/v3/projects/91f9212574384cac856b83b0e813f747/users/c75e234177fb4ed1a759e609b20da2c4/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:00.570258 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448daa7-f319-4abd-8a6a-e0c8102c4255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f9212574384cac856b83b0e813f747, user_id=c75e234177fb4ed1a759e609b20da2c4, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:00.572977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448daa7-f319-4abd-8a6a-e0c8102c4255 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:00.586621 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1227/2462] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:00 2022] PUT /identity/v3/projects/91f9212574384cac856b83b0e813f747/users/c75e234177fb4ed1a759e609b20da2c4/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:05:00.591640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448daa7-f319-4abd-8a6a-e0c8102c4255 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:00.607848 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf2624-59e9-4506-8d50-26c714bccf03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:00.609610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf2624-59e9-4506-8d50-26c714bccf03 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:00.630770 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1236/2463] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:00 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:05:00.680306 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448daa7-f319-4abd-8a6a-e0c8102c4255 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:00.685327 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41761f9d-812d-4101-aa46-6db9b96b78b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:00.685942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41761f9d-812d-4101-aa46-6db9b96b78b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:00.686253 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41761f9d-812d-4101-aa46-6db9b96b78b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:00.686553 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41761f9d-812d-4101-aa46-6db9b96b78b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:00.687230 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41761f9d-812d-4101-aa46-6db9b96b78b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:00.688819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41761f9d-812d-4101-aa46-6db9b96b78b2 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:00.707226 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1228/2464] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:05:00.708515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf2624-59e9-4506-8d50-26c714bccf03 service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:00.716501 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:00.723281 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf2624-59e9-4506-8d50-26c714bccf03 service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:00.728843 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540517ce-350f-4d11-b2b0-b7aa0e0900fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:00.729455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540517ce-350f-4d11-b2b0-b7aa0e0900fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:00.729743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540517ce-350f-4d11-b2b0-b7aa0e0900fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:00.730067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540517ce-350f-4d11-b2b0-b7aa0e0900fa admin admin] PATH_INFO: `/v3/projects/91f9212574384cac856b83b0e813f747/users/c75e234177fb4ed1a759e609b20da2c4/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:00.732573 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540517ce-350f-4d11-b2b0-b7aa0e0900fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f9212574384cac856b83b0e813f747, user_id=c75e234177fb4ed1a759e609b20da2c4, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:00.735301 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540517ce-350f-4d11-b2b0-b7aa0e0900fa admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:00.746423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1237/2465] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:00 2022] PUT /identity/v3/projects/91f9212574384cac856b83b0e813f747/users/c75e234177fb4ed1a759e609b20da2c4/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:00.780418 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41761f9d-812d-4101-aa46-6db9b96b78b2 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:00.790504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41761f9d-812d-4101-aa46-6db9b96b78b2 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:00.824274 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:00.857957 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9c203-ffcc-4400-ac1e-8d00e8bd59e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:00.858761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9c203-ffcc-4400-ac1e-8d00e8bd59e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:00.859166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9c203-ffcc-4400-ac1e-8d00e8bd59e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:00.859508 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9c203-ffcc-4400-ac1e-8d00e8bd59e7 admin admin] PATH_INFO: `/v3/users/7342d4cfe9e24b00a58a359ff320215f` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:00.860878 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9c203-ffcc-4400-ac1e-8d00e8bd59e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7342d4cfe9e24b00a58a359ff320215f)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:00.863095 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9c203-ffcc-4400-ac1e-8d00e8bd59e7 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:00.914542 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-27a9c203-ffcc-4400-ac1e-8d00e8bd59e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7342d4cfe9e24b00a58a359ff320215f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:00.923565 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-27a9c203-ffcc-4400-ac1e-8d00e8bd59e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7342d4cfe9e24b00a58a359ff320215f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:00.924312 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-27a9c203-ffcc-4400-ac1e-8d00e8bd59e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7342d4cfe9e24b00a58a359ff320215f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:00.939004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-27a9c203-ffcc-4400-ac1e-8d00e8bd59e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7342d4cfe9e24b00a58a359ff320215f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:00.939926 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-27a9c203-ffcc-4400-ac1e-8d00e8bd59e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7342d4cfe9e24b00a58a359ff320215f'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:00.954618 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1229/2466] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:05:00 2022] DELETE /identity/v3/users/7342d4cfe9e24b00a58a359ff320215f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:00.959096 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e98dec-3829-4e4a-81c3-4d45465c57b8 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:00.959344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e98dec-3829-4e4a-81c3-4d45465c57b8 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:00.959868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e98dec-3829-4e4a-81c3-4d45465c57b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:00.980435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-a5e98dec-3829-4e4a-81c3-4d45465c57b8 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:00.981680 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a5e98dec-3829-4e4a-81c3-4d45465c57b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:01.046808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1238/2467] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:01.056732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9c203-ffcc-4400-ac1e-8d00e8bd59e7 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:01.058307 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-27a9c203-ffcc-4400-ac1e-8d00e8bd59e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:01.140629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9c203-ffcc-4400-ac1e-8d00e8bd59e7 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:01.144591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f07374c-fb84-4392-8523-0f53c7f76844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:01.145205 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07374c-fb84-4392-8523-0f53c7f76844 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:01.145470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07374c-fb84-4392-8523-0f53c7f76844 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:01.145743 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07374c-fb84-4392-8523-0f53c7f76844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:01.162021 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f07374c-fb84-4392-8523-0f53c7f76844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:01.163681 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f07374c-fb84-4392-8523-0f53c7f76844 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:01.179182 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1230/2468] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:01.781951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e98dec-3829-4e4a-81c3-4d45465c57b8 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:01.785220 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a5e98dec-3829-4e4a-81c3-4d45465c57b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:01.815687 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e98dec-3829-4e4a-81c3-4d45465c57b8 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:01.821265 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea7100-1811-460a-8b66-71642eb5cee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:01.822071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea7100-1811-460a-8b66-71642eb5cee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:01.822457 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea7100-1811-460a-8b66-71642eb5cee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:01.822874 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea7100-1811-460a-8b66-71642eb5cee6 admin admin] PATH_INFO: `/v3/projects/465b27e83c83460393d42cd40d8db34a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:01.824286 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea7100-1811-460a-8b66-71642eb5cee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=465b27e83c83460393d42cd40d8db34a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:01.826564 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea7100-1811-460a-8b66-71642eb5cee6 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:01.884199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0aea7100-1811-460a-8b66-71642eb5cee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '465b27e83c83460393d42cd40d8db34a', 'request_id': 'req-0aea7100-1811-460a-8b66-71642eb5cee6'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:01.891944 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0aea7100-1811-460a-8b66-71642eb5cee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '465b27e83c83460393d42cd40d8db34a', 'request_id': 'req-0aea7100-1811-460a-8b66-71642eb5cee6'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:01.900939 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0aea7100-1811-460a-8b66-71642eb5cee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '465b27e83c83460393d42cd40d8db34a', 'request_id': 'req-0aea7100-1811-460a-8b66-71642eb5cee6'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:01.923879 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0aea7100-1811-460a-8b66-71642eb5cee6 admin admin] The token cache is being invalidate because project 465b27e83c83460393d42cd40d8db34a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:05:01.924600 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0aea7100-1811-460a-8b66-71642eb5cee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:01.927719 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0aea7100-1811-460a-8b66-71642eb5cee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:01.935477 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1239/2469] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:05:01 2022] DELETE /identity/v3/projects/465b27e83c83460393d42cd40d8db34a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:01.936451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f07374c-fb84-4392-8523-0f53c7f76844 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:01.960246 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f07374c-fb84-4392-8523-0f53c7f76844 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:02.062940 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:02.070288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79caee5-8867-4a34-8c64-1a92ee7aa3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:02.071518 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79caee5-8867-4a34-8c64-1a92ee7aa3f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:02.072657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79caee5-8867-4a34-8c64-1a92ee7aa3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:02.073157 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79caee5-8867-4a34-8c64-1a92ee7aa3f5 admin admin] PATH_INFO: `/v3/users/d3ba571dd5a34abca24d47fb2b466aa0` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:02.074573 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79caee5-8867-4a34-8c64-1a92ee7aa3f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3ba571dd5a34abca24d47fb2b466aa0)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:02.077051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79caee5-8867-4a34-8c64-1a92ee7aa3f5 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:02.152026 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d79caee5-8867-4a34-8c64-1a92ee7aa3f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3ba571dd5a34abca24d47fb2b466aa0'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:02.161722 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d79caee5-8867-4a34-8c64-1a92ee7aa3f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3ba571dd5a34abca24d47fb2b466aa0'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:02.162965 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d79caee5-8867-4a34-8c64-1a92ee7aa3f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3ba571dd5a34abca24d47fb2b466aa0'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:02.180760 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d79caee5-8867-4a34-8c64-1a92ee7aa3f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3ba571dd5a34abca24d47fb2b466aa0'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:02.181593 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d79caee5-8867-4a34-8c64-1a92ee7aa3f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3ba571dd5a34abca24d47fb2b466aa0'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:02.199759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1231/2470] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:05:01 2022] DELETE /identity/v3/users/d3ba571dd5a34abca24d47fb2b466aa0 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:05:02.534755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea7100-1811-460a-8b66-71642eb5cee6 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:02.536276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0aea7100-1811-460a-8b66-71642eb5cee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:02.556093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea7100-1811-460a-8b66-71642eb5cee6 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:02.622673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5b54c1-1025-4284-9da2-133f62f98bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:02.623431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b54c1-1025-4284-9da2-133f62f98bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:02.624103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b54c1-1025-4284-9da2-133f62f98bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:02.624444 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5b54c1-1025-4284-9da2-133f62f98bc0 admin admin] PATH_INFO: `/v3/projects/4dff1e9ac01d427cbc579365ce7ccd4e` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:02.625750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b54c1-1025-4284-9da2-133f62f98bc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dff1e9ac01d427cbc579365ce7ccd4e)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:02.627879 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5b54c1-1025-4284-9da2-133f62f98bc0 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:02.655000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:02.690448 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ea5b54c1-1025-4284-9da2-133f62f98bc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dff1e9ac01d427cbc579365ce7ccd4e', 'request_id': 'req-ea5b54c1-1025-4284-9da2-133f62f98bc0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:02.697332 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ea5b54c1-1025-4284-9da2-133f62f98bc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dff1e9ac01d427cbc579365ce7ccd4e', 'request_id': 'req-ea5b54c1-1025-4284-9da2-133f62f98bc0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:02.704405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ea5b54c1-1025-4284-9da2-133f62f98bc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dff1e9ac01d427cbc579365ce7ccd4e', 'request_id': 'req-ea5b54c1-1025-4284-9da2-133f62f98bc0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:02.722034 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ea5b54c1-1025-4284-9da2-133f62f98bc0 admin admin] The token cache is being invalidate because project 4dff1e9ac01d427cbc579365ce7ccd4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:05:02.722327 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ea5b54c1-1025-4284-9da2-133f62f98bc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:02.723014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-ea5b54c1-1025-4284-9da2-133f62f98bc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:02.727861 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1240/2471] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:05:02 2022] DELETE /identity/v3/projects/4dff1e9ac01d427cbc579365ce7ccd4e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:02.731580 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b7e54-9996-4622-9e46-8b8a822c0db3 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:02.732674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b7e54-9996-4622-9e46-8b8a822c0db3 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:02.733171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b7e54-9996-4622-9e46-8b8a822c0db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:02.755199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-223b7e54-9996-4622-9e46-8b8a822c0db3 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:02.756579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-223b7e54-9996-4622-9e46-8b8a822c0db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:02.849528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1232/2472] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:02.857943 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb5fe2-3c81-4a5f-b675-584cf72274d9 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:02.858229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb5fe2-3c81-4a5f-b675-584cf72274d9 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:02.858510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb5fe2-3c81-4a5f-b675-584cf72274d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:02.879956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-69eb5fe2-3c81-4a5f-b675-584cf72274d9 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:02.881250 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-69eb5fe2-3c81-4a5f-b675-584cf72274d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:02.891721 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1241/2473] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:02.901528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b7e54-9996-4622-9e46-8b8a822c0db3 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:02.913326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b7e54-9996-4622-9e46-8b8a822c0db3 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:02.918606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa4c3e-bf50-48d1-8a35-9722864a1ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:02.919432 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa4c3e-bf50-48d1-8a35-9722864a1ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:02.919784 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa4c3e-bf50-48d1-8a35-9722864a1ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:02.920124 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa4c3e-bf50-48d1-8a35-9722864a1ce6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:02.920908 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa4c3e-bf50-48d1-8a35-9722864a1ce6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:05:02.921291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa4c3e-bf50-48d1-8a35-9722864a1ce6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:02.923168 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa4c3e-bf50-48d1-8a35-9722864a1ce6 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:02.943195 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1233/2474] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:05:02.944917 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb5fe2-3c81-4a5f-b675-584cf72274d9 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:02.956820 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb5fe2-3c81-4a5f-b675-584cf72274d9 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:02.963517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98667af-45cd-44d8-bdac-d2dee3a4b222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:02.964294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98667af-45cd-44d8-bdac-d2dee3a4b222 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:02.965340 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98667af-45cd-44d8-bdac-d2dee3a4b222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:02.965862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98667af-45cd-44d8-bdac-d2dee3a4b222 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:02.966721 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98667af-45cd-44d8-bdac-d2dee3a4b222 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:02.969578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98667af-45cd-44d8-bdac-d2dee3a4b222 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:03.014982 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1242/2475] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:05:02 2022] POST /identity/v3/projects => generated 407 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:03.016130 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa4c3e-bf50-48d1-8a35-9722864a1ce6 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:03.027668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa4c3e-bf50-48d1-8a35-9722864a1ce6 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:03.133599 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc35402-a416-46da-bd76-d0b16d4bd948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:03.134355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc35402-a416-46da-bd76-d0b16d4bd948 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:03.137235 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc35402-a416-46da-bd76-d0b16d4bd948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:03.138413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc35402-a416-46da-bd76-d0b16d4bd948 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:03.140154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc35402-a416-46da-bd76-d0b16d4bd948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:03.142891 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc35402-a416-46da-bd76-d0b16d4bd948 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:03.158209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:03.174710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1234/2476] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:05:03 2022] POST /identity/v3/users => generated 304 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:05:03.175994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98667af-45cd-44d8-bdac-d2dee3a4b222 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:03.189302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98667af-45cd-44d8-bdac-d2dee3a4b222 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:03.195113 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397cfec4-d8d9-40dd-be07-0f574f0b8a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:03.195866 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cfec4-d8d9-40dd-be07-0f574f0b8a87 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:03.196053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cfec4-d8d9-40dd-be07-0f574f0b8a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:03.196284 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cfec4-d8d9-40dd-be07-0f574f0b8a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:03.196970 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397cfec4-d8d9-40dd-be07-0f574f0b8a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:03.198614 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397cfec4-d8d9-40dd-be07-0f574f0b8a87 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:03.211546 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1243/2477] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:05:03.215668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc35402-a416-46da-bd76-d0b16d4bd948 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:03.228057 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc35402-a416-46da-bd76-d0b16d4bd948 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:03.235889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a771ee-e66f-4fee-a412-0f87e2e05b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:03.236677 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a771ee-e66f-4fee-a412-0f87e2e05b1e admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:03.237056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a771ee-e66f-4fee-a412-0f87e2e05b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:03.237433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a771ee-e66f-4fee-a412-0f87e2e05b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:03.238240 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a771ee-e66f-4fee-a412-0f87e2e05b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:03.241634 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a771ee-e66f-4fee-a412-0f87e2e05b1e admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:03.253504 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1235/2478] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:03.257487 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397cfec4-d8d9-40dd-be07-0f574f0b8a87 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:03.268882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397cfec4-d8d9-40dd-be07-0f574f0b8a87 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:03.273906 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d9a09-f204-45cf-adf3-f05c47c491be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:03.274602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d9a09-f204-45cf-adf3-f05c47c491be admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:03.274915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d9a09-f204-45cf-adf3-f05c47c491be admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:03.275331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d9a09-f204-45cf-adf3-f05c47c491be admin admin] PATH_INFO: `/v3/projects/a4d553df75a14c5ab1010ad221ac7825/users/adab09127dfe46628c4a640a82c53360/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:03.291381 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d9a09-f204-45cf-adf3-f05c47c491be admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4d553df75a14c5ab1010ad221ac7825, user_id=adab09127dfe46628c4a640a82c53360, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:03.294593 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d9a09-f204-45cf-adf3-f05c47c491be admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:03.336521 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1244/2479] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:03 2022] PUT /identity/v3/projects/a4d553df75a14c5ab1010ad221ac7825/users/adab09127dfe46628c4a640a82c53360/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:05:03.349485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e72b-710e-4e0d-97d5-8f5615c32856 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:03.349893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e72b-710e-4e0d-97d5-8f5615c32856 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:03.350412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e72b-710e-4e0d-97d5-8f5615c32856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:03.392118 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-5858e72b-710e-4e0d-97d5-8f5615c32856 None None] MFA Rules not processed for user `adab09127dfe46628c4a640a82c53360`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:03.393707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-5858e72b-710e-4e0d-97d5-8f5615c32856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:03.482400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1236/2480] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:03 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:03.487205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d9a09-f204-45cf-adf3-f05c47c491be admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:03.589407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d9a09-f204-45cf-adf3-f05c47c491be admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:03.595486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fde9f-6eec-4b28-85eb-823345b4fe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:03.596344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fde9f-6eec-4b28-85eb-823345b4fe60 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:03.596623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fde9f-6eec-4b28-85eb-823345b4fe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:03.597214 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fde9f-6eec-4b28-85eb-823345b4fe60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:03.597887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fde9f-6eec-4b28-85eb-823345b4fe60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:03.600150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fde9f-6eec-4b28-85eb-823345b4fe60 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:03.635701 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1245/2481] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:05:03 2022] POST /identity/v3/projects => generated 407 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:03.637803 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858e72b-710e-4e0d-97d5-8f5615c32856 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:03.649784 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858e72b-710e-4e0d-97d5-8f5615c32856 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:03.815474 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd1fb0-d46c-4f2e-a735-149099278f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:03.816611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd1fb0-d46c-4f2e-a735-149099278f25 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:03.817159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd1fb0-d46c-4f2e-a735-149099278f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:03.817638 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd1fb0-d46c-4f2e-a735-149099278f25 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:03.818493 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd1fb0-d46c-4f2e-a735-149099278f25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:03.820969 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd1fb0-d46c-4f2e-a735-149099278f25 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:03.871320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1237/2482] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:05:03 2022] POST /identity/v3/users => generated 310 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:03.876099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fde9f-6eec-4b28-85eb-823345b4fe60 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:03.892849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fde9f-6eec-4b28-85eb-823345b4fe60 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:03.907603 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819f8c7-e9d7-404f-abae-3b9dec4c215a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:03.909018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819f8c7-e9d7-404f-abae-3b9dec4c215a admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:03.909573 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819f8c7-e9d7-404f-abae-3b9dec4c215a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:03.910010 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819f8c7-e9d7-404f-abae-3b9dec4c215a admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:03.910855 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819f8c7-e9d7-404f-abae-3b9dec4c215a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:03.912644 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819f8c7-e9d7-404f-abae-3b9dec4c215a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:03.941847 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1246/2483] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:03.946459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd1fb0-d46c-4f2e-a735-149099278f25 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:03.963558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd1fb0-d46c-4f2e-a735-149099278f25 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:03.976108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0d8a2-1342-4d89-8eaa-8287ced2ecde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:03.977045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d8a2-1342-4d89-8eaa-8287ced2ecde admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:03.977438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d8a2-1342-4d89-8eaa-8287ced2ecde admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:03.979125 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d8a2-1342-4d89-8eaa-8287ced2ecde admin admin] PATH_INFO: `/v3/projects/967000d73ae44e57ae8a3903ffd88b3a/users/80eaddcfaf6d41c1b95447d793c66af9/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:04.015973 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0d8a2-1342-4d89-8eaa-8287ced2ecde admin admin] RBAC: Authorizing `identity:create_grant(project_id=967000d73ae44e57ae8a3903ffd88b3a, user_id=80eaddcfaf6d41c1b95447d793c66af9, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:04.018770 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0d8a2-1342-4d89-8eaa-8287ced2ecde admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:04.047297 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1238/2484] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:03 2022] PUT /identity/v3/projects/967000d73ae44e57ae8a3903ffd88b3a/users/80eaddcfaf6d41c1b95447d793c66af9/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:04.051091 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819f8c7-e9d7-404f-abae-3b9dec4c215a admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:04.066022 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819f8c7-e9d7-404f-abae-3b9dec4c215a admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:04.084033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:04.200501 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1b1f4-b0b0-4cf5-8c48-2c1d9522e26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:04.201397 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1b1f4-b0b0-4cf5-8c48-2c1d9522e26e admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:04.201775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1b1f4-b0b0-4cf5-8c48-2c1d9522e26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:04.202145 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1b1f4-b0b0-4cf5-8c48-2c1d9522e26e admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:04.202971 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1b1f4-b0b0-4cf5-8c48-2c1d9522e26e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:04.205004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1b1f4-b0b0-4cf5-8c48-2c1d9522e26e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:04.228930 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1247/2485] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:04.233522 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0d8a2-1342-4d89-8eaa-8287ced2ecde admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:04.244937 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0d8a2-1342-4d89-8eaa-8287ced2ecde admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:04.250660 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc332b83-6baf-4d5b-825f-260c7d291cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:04.251755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc332b83-6baf-4d5b-825f-260c7d291cdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:04.251755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc332b83-6baf-4d5b-825f-260c7d291cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:04.251985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc332b83-6baf-4d5b-825f-260c7d291cdb admin admin] PATH_INFO: `/v3/projects/967000d73ae44e57ae8a3903ffd88b3a/users/80eaddcfaf6d41c1b95447d793c66af9/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:04.254551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc332b83-6baf-4d5b-825f-260c7d291cdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=967000d73ae44e57ae8a3903ffd88b3a, user_id=80eaddcfaf6d41c1b95447d793c66af9, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:04.257313 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc332b83-6baf-4d5b-825f-260c7d291cdb admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:04.271855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1239/2486] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:04 2022] PUT /identity/v3/projects/967000d73ae44e57ae8a3903ffd88b3a/users/80eaddcfaf6d41c1b95447d793c66af9/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:05:04.283373 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd7c75-d64c-4f7a-9646-ab16ebdea239 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:04.284244 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd7c75-d64c-4f7a-9646-ab16ebdea239 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:04.284626 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd7c75-d64c-4f7a-9646-ab16ebdea239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:04.320981 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-9dcd7c75-d64c-4f7a-9646-ab16ebdea239 None None] MFA Rules not processed for user `80eaddcfaf6d41c1b95447d793c66af9`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:04.323687 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-9dcd7c75-d64c-4f7a-9646-ab16ebdea239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:04.433948 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1248/2487] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:04 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:04.442371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc332b83-6baf-4d5b-825f-260c7d291cdb admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:04.445713 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-bc332b83-6baf-4d5b-825f-260c7d291cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:04.497615 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:04.564447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc332b83-6baf-4d5b-825f-260c7d291cdb admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:04.574170 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1237a-9e19-4ffa-977f-52c9f1496970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:04.574839 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1237a-9e19-4ffa-977f-52c9f1496970 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:04.575189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1237a-9e19-4ffa-977f-52c9f1496970 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:04.575958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1237a-9e19-4ffa-977f-52c9f1496970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:04.603147 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1237a-9e19-4ffa-977f-52c9f1496970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:04.607845 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1237a-9e19-4ffa-977f-52c9f1496970 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:04.646031 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1240/2488] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:04 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:05.003535 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd7c75-d64c-4f7a-9646-ab16ebdea239 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:05.017737 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd7c75-d64c-4f7a-9646-ab16ebdea239 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:05.023296 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bddd96-4c59-4bc8-a9dd-a76189a3a2c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:05.024613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bddd96-4c59-4bc8-a9dd-a76189a3a2c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:05.024958 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bddd96-4c59-4bc8-a9dd-a76189a3a2c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:05.025287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bddd96-4c59-4bc8-a9dd-a76189a3a2c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:05.148084 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bddd96-4c59-4bc8-a9dd-a76189a3a2c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:05.149594 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bddd96-4c59-4bc8-a9dd-a76189a3a2c8 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:05.175181 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1249/2489] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:05 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:06.465326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298be70-118c-4d1b-b77c-b328228d5ea5 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:06.465924 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298be70-118c-4d1b-b77c-b328228d5ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:06.466056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298be70-118c-4d1b-b77c-b328228d5ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:06.495676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-1298be70-118c-4d1b-b77c-b328228d5ea5 None None] MFA Rules not processed for user `c75e234177fb4ed1a759e609b20da2c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:06.497559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-1298be70-118c-4d1b-b77c-b328228d5ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:06.573317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1241/2490] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:06.586054 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bddd96-4c59-4bc8-a9dd-a76189a3a2c8 service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:06.587556 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-07bddd96-4c59-4bc8-a9dd-a76189a3a2c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:06.631887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:06.650450 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bddd96-4c59-4bc8-a9dd-a76189a3a2c8 service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:06.654696 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16a32c-7eb4-4780-b834-6aa6f444565b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5248cb1c39412680cce0d9891fe1ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:06.655288 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16a32c-7eb4-4780-b834-6aa6f444565b service glance] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:06.655564 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16a32c-7eb4-4780-b834-6aa6f444565b service glance] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:06.655833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16a32c-7eb4-4780-b834-6aa6f444565b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:06.657637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-aa16a32c-7eb4-4780-b834-6aa6f444565b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:06.721936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16a32c-7eb4-4780-b834-6aa6f444565b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:06.723421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16a32c-7eb4-4780-b834-6aa6f444565b service glance] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:06.733588 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1250/2491] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:06.875469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5139f4-cdd9-4338-9177-7670ce3397ad None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:06.875786 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5139f4-cdd9-4338-9177-7670ce3397ad None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:06.876067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5139f4-cdd9-4338-9177-7670ce3397ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:06.896377 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-5f5139f4-cdd9-4338-9177-7670ce3397ad None None] MFA Rules not processed for user `6256407d9df648d5bbe33495cb9a8b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:06.897597 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-5f5139f4-cdd9-4338-9177-7670ce3397ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:06.973278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1242/2492] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Sep 22 15:05:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:07.001880 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16a32c-7eb4-4780-b834-6aa6f444565b service glance] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:07.003932 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-aa16a32c-7eb4-4780-b834-6aa6f444565b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:07.081549 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16a32c-7eb4-4780-b834-6aa6f444565b service glance] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:07.088005 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea5cc3-ab87-4034-8b94-1d64a8ac2e5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a265623ae734936a340c97df852e227', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:07.088617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea5cc3-ab87-4034-8b94-1d64a8ac2e5e service swift] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:07.089095 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea5cc3-ab87-4034-8b94-1d64a8ac2e5e service swift] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:07.089387 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea5cc3-ab87-4034-8b94-1d64a8ac2e5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:07.100876 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea5cc3-ab87-4034-8b94-1d64a8ac2e5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:07.102489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea5cc3-ab87-4034-8b94-1d64a8ac2e5e service swift] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:07.111609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1251/2493] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Sep 22 15:05:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:05:14.451797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5139f4-cdd9-4338-9177-7670ce3397ad None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:14.532038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:14.536658 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5139f4-cdd9-4338-9177-7670ce3397ad None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:14.542135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ac690-3c93-45cd-8568-4e919f5ff913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:14.542826 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ac690-3c93-45cd-8568-4e919f5ff913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:14.544209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ac690-3c93-45cd-8568-4e919f5ff913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:14.544507 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ac690-3c93-45cd-8568-4e919f5ff913 admin admin] PATH_INFO: `/v3/users/adab09127dfe46628c4a640a82c53360` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:14.545718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ac690-3c93-45cd-8568-4e919f5ff913 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adab09127dfe46628c4a640a82c53360)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:14.547652 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ac690-3c93-45cd-8568-4e919f5ff913 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:14.600071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b94ac690-3c93-45cd-8568-4e919f5ff913 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adab09127dfe46628c4a640a82c53360'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:14.625829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b94ac690-3c93-45cd-8568-4e919f5ff913 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adab09127dfe46628c4a640a82c53360'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:14.629381 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b94ac690-3c93-45cd-8568-4e919f5ff913 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adab09127dfe46628c4a640a82c53360'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:14.645705 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b94ac690-3c93-45cd-8568-4e919f5ff913 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adab09127dfe46628c4a640a82c53360'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:14.646489 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b94ac690-3c93-45cd-8568-4e919f5ff913 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adab09127dfe46628c4a640a82c53360'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:14.664448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1243/2494] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:05:14 2022] DELETE /identity/v3/users/adab09127dfe46628c4a640a82c53360 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:14.669267 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2346aa3-d729-4aa8-95df-03ba2d0744ec None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:14.669761 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2346aa3-d729-4aa8-95df-03ba2d0744ec None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:14.669889 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2346aa3-d729-4aa8-95df-03ba2d0744ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:14.694261 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-c2346aa3-d729-4aa8-95df-03ba2d0744ec None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:14.695608 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c2346aa3-d729-4aa8-95df-03ba2d0744ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:14.791014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1252/2495] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:14.803654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ac690-3c93-45cd-8568-4e919f5ff913 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:14.805979 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b94ac690-3c93-45cd-8568-4e919f5ff913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:14.885805 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ac690-3c93-45cd-8568-4e919f5ff913 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:14.889519 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f28191-85ac-4474-9c41-58da992d26d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:14.890129 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f28191-85ac-4474-9c41-58da992d26d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:14.890414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f28191-85ac-4474-9c41-58da992d26d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:14.890678 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f28191-85ac-4474-9c41-58da992d26d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:14.903260 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f28191-85ac-4474-9c41-58da992d26d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:14.904838 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f28191-85ac-4474-9c41-58da992d26d0 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:14.916984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1244/2496] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:15.264693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2346aa3-d729-4aa8-95df-03ba2d0744ec None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:15.266139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c2346aa3-d729-4aa8-95df-03ba2d0744ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:15.288996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2346aa3-d729-4aa8-95df-03ba2d0744ec None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:15.294394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e4e31-2b9e-4552-9aaf-691b243f72c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:15.295039 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e4e31-2b9e-4552-9aaf-691b243f72c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:15.295316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e4e31-2b9e-4552-9aaf-691b243f72c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:15.295590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e4e31-2b9e-4552-9aaf-691b243f72c7 admin admin] PATH_INFO: `/v3/projects/a4d553df75a14c5ab1010ad221ac7825` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:15.296741 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e4e31-2b9e-4552-9aaf-691b243f72c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4d553df75a14c5ab1010ad221ac7825)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:15.298872 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e4e31-2b9e-4552-9aaf-691b243f72c7 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:15.325455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:15.370136 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-d51e4e31-2b9e-4552-9aaf-691b243f72c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4d553df75a14c5ab1010ad221ac7825', 'request_id': 'req-d51e4e31-2b9e-4552-9aaf-691b243f72c7'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:15.378768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-d51e4e31-2b9e-4552-9aaf-691b243f72c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4d553df75a14c5ab1010ad221ac7825', 'request_id': 'req-d51e4e31-2b9e-4552-9aaf-691b243f72c7'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:15.387681 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-d51e4e31-2b9e-4552-9aaf-691b243f72c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4d553df75a14c5ab1010ad221ac7825', 'request_id': 'req-d51e4e31-2b9e-4552-9aaf-691b243f72c7'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:15.407169 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-d51e4e31-2b9e-4552-9aaf-691b243f72c7 admin admin] The token cache is being invalidate because project a4d553df75a14c5ab1010ad221ac7825 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:05:15.407547 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-d51e4e31-2b9e-4552-9aaf-691b243f72c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:15.408254 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-d51e4e31-2b9e-4552-9aaf-691b243f72c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:15.409849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1253/2497] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:05:15 2022] DELETE /identity/v3/projects/a4d553df75a14c5ab1010ad221ac7825 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:15.413911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f28191-85ac-4474-9c41-58da992d26d0 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:15.425736 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f28191-85ac-4474-9c41-58da992d26d0 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:15.499222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdbadd-26e8-4003-86c4-43906642b194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:15.500687 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdbadd-26e8-4003-86c4-43906642b194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:15.501131 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdbadd-26e8-4003-86c4-43906642b194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:15.501457 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdbadd-26e8-4003-86c4-43906642b194 admin admin] PATH_INFO: `/v3/users/80eaddcfaf6d41c1b95447d793c66af9` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:15.502615 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdbadd-26e8-4003-86c4-43906642b194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80eaddcfaf6d41c1b95447d793c66af9)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:15.504562 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdbadd-26e8-4003-86c4-43906642b194 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:15.536304 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:15.578982 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-1bbdbadd-26e8-4003-86c4-43906642b194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80eaddcfaf6d41c1b95447d793c66af9'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:15.589808 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-1bbdbadd-26e8-4003-86c4-43906642b194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80eaddcfaf6d41c1b95447d793c66af9'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:15.591412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-1bbdbadd-26e8-4003-86c4-43906642b194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80eaddcfaf6d41c1b95447d793c66af9'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:15.607821 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-1bbdbadd-26e8-4003-86c4-43906642b194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80eaddcfaf6d41c1b95447d793c66af9'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:15.608537 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-1bbdbadd-26e8-4003-86c4-43906642b194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80eaddcfaf6d41c1b95447d793c66af9'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:15.623360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1245/2498] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:05:15 2022] DELETE /identity/v3/users/80eaddcfaf6d41c1b95447d793c66af9 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:15.974563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e4e31-2b9e-4552-9aaf-691b243f72c7 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:15.976356 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d51e4e31-2b9e-4552-9aaf-691b243f72c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:15.998022 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e4e31-2b9e-4552-9aaf-691b243f72c7 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:16.086590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975682b-0f28-42f2-84d6-e200786bc0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:16.087531 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975682b-0f28-42f2-84d6-e200786bc0fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:16.087808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975682b-0f28-42f2-84d6-e200786bc0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:16.088077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975682b-0f28-42f2-84d6-e200786bc0fd admin admin] PATH_INFO: `/v3/projects/967000d73ae44e57ae8a3903ffd88b3a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:16.089322 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975682b-0f28-42f2-84d6-e200786bc0fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=967000d73ae44e57ae8a3903ffd88b3a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:16.091315 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975682b-0f28-42f2-84d6-e200786bc0fd admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:16.156875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3975682b-0f28-42f2-84d6-e200786bc0fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '967000d73ae44e57ae8a3903ffd88b3a', 'request_id': 'req-3975682b-0f28-42f2-84d6-e200786bc0fd'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:16.163978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3975682b-0f28-42f2-84d6-e200786bc0fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '967000d73ae44e57ae8a3903ffd88b3a', 'request_id': 'req-3975682b-0f28-42f2-84d6-e200786bc0fd'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:16.172563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3975682b-0f28-42f2-84d6-e200786bc0fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '967000d73ae44e57ae8a3903ffd88b3a', 'request_id': 'req-3975682b-0f28-42f2-84d6-e200786bc0fd'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:16.187301 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3975682b-0f28-42f2-84d6-e200786bc0fd admin admin] The token cache is being invalidate because project 967000d73ae44e57ae8a3903ffd88b3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:05:16.187586 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3975682b-0f28-42f2-84d6-e200786bc0fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:16.188202 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3975682b-0f28-42f2-84d6-e200786bc0fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:16.190039 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1254/2499] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:05:15 2022] DELETE /identity/v3/projects/967000d73ae44e57ae8a3903ffd88b3a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:16.194926 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a25b4-4ea1-4a40-a241-3764e2469542 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:16.195153 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a25b4-4ea1-4a40-a241-3764e2469542 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:16.195433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a25b4-4ea1-4a40-a241-3764e2469542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:16.217257 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-5c3a25b4-4ea1-4a40-a241-3764e2469542 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:16.218495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-5c3a25b4-4ea1-4a40-a241-3764e2469542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:16.297621 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1246/2500] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:16.305467 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00ede6-7ba1-4e6e-bcc8-7bab4b36d5bf None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:16.305755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00ede6-7ba1-4e6e-bcc8-7bab4b36d5bf None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:16.306032 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00ede6-7ba1-4e6e-bcc8-7bab4b36d5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:16.336846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-cd00ede6-7ba1-4e6e-bcc8-7bab4b36d5bf None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:16.338089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-cd00ede6-7ba1-4e6e-bcc8-7bab4b36d5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:16.351418 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1255/2501] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:16.352838 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a25b4-4ea1-4a40-a241-3764e2469542 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:16.364255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a25b4-4ea1-4a40-a241-3764e2469542 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:16.369239 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c7b9a-6579-4c95-a71b-25cbf82bfa3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:16.369883 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c7b9a-6579-4c95-a71b-25cbf82bfa3a admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:16.370218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c7b9a-6579-4c95-a71b-25cbf82bfa3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:16.370433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c7b9a-6579-4c95-a71b-25cbf82bfa3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:16.371144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c7b9a-6579-4c95-a71b-25cbf82bfa3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:05:16.371450 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c7b9a-6579-4c95-a71b-25cbf82bfa3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:16.373053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c7b9a-6579-4c95-a71b-25cbf82bfa3a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:16.385523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1247/2502] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:05:16.390073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00ede6-7ba1-4e6e-bcc8-7bab4b36d5bf None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:16.399994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00ede6-7ba1-4e6e-bcc8-7bab4b36d5bf None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:16.405352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c6834-0ae1-42e9-87b8-b76606cddc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:16.405949 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c6834-0ae1-42e9-87b8-b76606cddc50 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:16.406225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c6834-0ae1-42e9-87b8-b76606cddc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:16.406485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c6834-0ae1-42e9-87b8-b76606cddc50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:16.407140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c6834-0ae1-42e9-87b8-b76606cddc50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:16.409022 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2c6834-0ae1-42e9-87b8-b76606cddc50 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:16.442806 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1256/2503] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:05:16 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:16.443864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c7b9a-6579-4c95-a71b-25cbf82bfa3a admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:16.454985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c7b9a-6579-4c95-a71b-25cbf82bfa3a admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:16.517868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69879c72-dc54-4f67-9ac3-9dd07971048a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:16.518579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69879c72-dc54-4f67-9ac3-9dd07971048a admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:16.518797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69879c72-dc54-4f67-9ac3-9dd07971048a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:16.519076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69879c72-dc54-4f67-9ac3-9dd07971048a admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:16.519731 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69879c72-dc54-4f67-9ac3-9dd07971048a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:16.521778 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69879c72-dc54-4f67-9ac3-9dd07971048a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:16.559925 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1248/2504] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:05:16 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:05:16.563281 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c6834-0ae1-42e9-87b8-b76606cddc50 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:16.570363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:16.575034 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c6834-0ae1-42e9-87b8-b76606cddc50 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:16.587372 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc8189-98db-43eb-8925-f3d7a3837639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:16.588043 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc8189-98db-43eb-8925-f3d7a3837639 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:16.588324 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc8189-98db-43eb-8925-f3d7a3837639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:16.588975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc8189-98db-43eb-8925-f3d7a3837639 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:16.589288 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc8189-98db-43eb-8925-f3d7a3837639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:16.591529 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc8189-98db-43eb-8925-f3d7a3837639 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:16.602775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1257/2505] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:05:16.606845 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69879c72-dc54-4f67-9ac3-9dd07971048a admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:16.619954 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69879c72-dc54-4f67-9ac3-9dd07971048a admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:16.624993 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85f8f7-8866-4dc8-bdc6-7810d6e5820a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:16.625615 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85f8f7-8866-4dc8-bdc6-7810d6e5820a admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:16.625895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85f8f7-8866-4dc8-bdc6-7810d6e5820a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:16.626170 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85f8f7-8866-4dc8-bdc6-7810d6e5820a admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:16.626861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85f8f7-8866-4dc8-bdc6-7810d6e5820a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:16.628457 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85f8f7-8866-4dc8-bdc6-7810d6e5820a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:16.639816 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1249/2506] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:16.644414 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc8189-98db-43eb-8925-f3d7a3837639 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:16.656219 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc8189-98db-43eb-8925-f3d7a3837639 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:16.660466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8502b9-38d7-432a-9d4f-2eebfdbd726d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:16.661129 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8502b9-38d7-432a-9d4f-2eebfdbd726d admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:16.661411 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8502b9-38d7-432a-9d4f-2eebfdbd726d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:16.661682 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8502b9-38d7-432a-9d4f-2eebfdbd726d admin admin] PATH_INFO: `/v3/projects/b5213f7f369b41059977afd16a0eac64/users/c4c567514ee5472b943d0f7ae35ed6cc/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:16.677128 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8502b9-38d7-432a-9d4f-2eebfdbd726d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5213f7f369b41059977afd16a0eac64, user_id=c4c567514ee5472b943d0f7ae35ed6cc, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:16.679834 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8502b9-38d7-432a-9d4f-2eebfdbd726d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:16.693598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1258/2507] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:16 2022] PUT /identity/v3/projects/b5213f7f369b41059977afd16a0eac64/users/c4c567514ee5472b943d0f7ae35ed6cc/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:16.705304 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a991d-08bc-4915-bdde-53b571b9f88b None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:16.705614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a991d-08bc-4915-bdde-53b571b9f88b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:16.705948 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a991d-08bc-4915-bdde-53b571b9f88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:16.713872 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282ac90-1b9f-4786-82b2-fc2f3fd02b19 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:16.714786 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282ac90-1b9f-4786-82b2-fc2f3fd02b19 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:16.714935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:16.715111 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282ac90-1b9f-4786-82b2-fc2f3fd02b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:16.735050 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-1282ac90-1b9f-4786-82b2-fc2f3fd02b19 None None] MFA Rules not processed for user `59c1a8f322724f6490fe244c2704e1dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:16.736274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1282ac90-1b9f-4786-82b2-fc2f3fd02b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:16.737025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-387a991d-08bc-4915-bdde-53b571b9f88b None None] MFA Rules not processed for user `c4c567514ee5472b943d0f7ae35ed6cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:16.738204 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-387a991d-08bc-4915-bdde-53b571b9f88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:16.794905 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1259/2508] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:05:16.802811 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1250/2509] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:16.807111 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282ac90-1b9f-4786-82b2-fc2f3fd02b19 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:16.876101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282ac90-1b9f-4786-82b2-fc2f3fd02b19 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:16.880600 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b83417-b638-4084-9c0a-aaefe1405583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:16.881233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b83417-b638-4084-9c0a-aaefe1405583 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:16.881507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b83417-b638-4084-9c0a-aaefe1405583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:16.881774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b83417-b638-4084-9c0a-aaefe1405583 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:16.882402 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b83417-b638-4084-9c0a-aaefe1405583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:16.884329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b83417-b638-4084-9c0a-aaefe1405583 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:16.913091 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1260/2510] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:05:16 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:16.917216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a991d-08bc-4915-bdde-53b571b9f88b None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:16.928310 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a991d-08bc-4915-bdde-53b571b9f88b None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:16.988961 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e086f6c-fa42-4f96-bfef-3d58073cf7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:16.989647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e086f6c-fa42-4f96-bfef-3d58073cf7a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:16.989884 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e086f6c-fa42-4f96-bfef-3d58073cf7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:16.990133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e086f6c-fa42-4f96-bfef-3d58073cf7a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:16.990815 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e086f6c-fa42-4f96-bfef-3d58073cf7a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:16.992822 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e086f6c-fa42-4f96-bfef-3d58073cf7a1 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:17.018279 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1251/2511] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:05:16 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:05:17.022297 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b83417-b638-4084-9c0a-aaefe1405583 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:17.033928 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b83417-b638-4084-9c0a-aaefe1405583 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:17.038920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad711f-1317-4fab-9b9a-994623830b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:17.039537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad711f-1317-4fab-9b9a-994623830b1e admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:17.039844 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad711f-1317-4fab-9b9a-994623830b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:17.040116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad711f-1317-4fab-9b9a-994623830b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:17.040795 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad711f-1317-4fab-9b9a-994623830b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:17.042454 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad711f-1317-4fab-9b9a-994623830b1e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:17.053123 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1261/2512] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:17.057364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e086f6c-fa42-4f96-bfef-3d58073cf7a1 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:17.067741 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e086f6c-fa42-4f96-bfef-3d58073cf7a1 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:17.073034 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c9b67-3664-4d2a-a8be-8fa615ffa93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:17.073701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c9b67-3664-4d2a-a8be-8fa615ffa93e admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:17.074014 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c9b67-3664-4d2a-a8be-8fa615ffa93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:17.074293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c9b67-3664-4d2a-a8be-8fa615ffa93e admin admin] PATH_INFO: `/v3/projects/444a8b826298487584ddaf3d2122b06a/users/8b4d9bbd0db54bb699cfd642c1a07242/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:17.093606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c9b67-3664-4d2a-a8be-8fa615ffa93e admin admin] RBAC: Authorizing `identity:create_grant(project_id=444a8b826298487584ddaf3d2122b06a, user_id=8b4d9bbd0db54bb699cfd642c1a07242, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:17.096299 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c9b67-3664-4d2a-a8be-8fa615ffa93e admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:17.110874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1252/2513] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:17 2022] PUT /identity/v3/projects/444a8b826298487584ddaf3d2122b06a/users/8b4d9bbd0db54bb699cfd642c1a07242/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:17.115298 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad711f-1317-4fab-9b9a-994623830b1e admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:17.127518 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad711f-1317-4fab-9b9a-994623830b1e admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:17.176407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:17.190446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ca17e-a80a-4dfd-822d-2ad93ca2b7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:17.191421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ca17e-a80a-4dfd-822d-2ad93ca2b7be admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:17.191421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ca17e-a80a-4dfd-822d-2ad93ca2b7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:17.191710 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ca17e-a80a-4dfd-822d-2ad93ca2b7be admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:17.192353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ca17e-a80a-4dfd-822d-2ad93ca2b7be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:17.193941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ca17e-a80a-4dfd-822d-2ad93ca2b7be admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:17.207859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1262/2514] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:17.209579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c9b67-3664-4d2a-a8be-8fa615ffa93e admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:17.219978 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c9b67-3664-4d2a-a8be-8fa615ffa93e admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:17.224669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd79b9-01f8-4187-9e57-96f6ea46fed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:17.225346 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd79b9-01f8-4187-9e57-96f6ea46fed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:17.225659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd79b9-01f8-4187-9e57-96f6ea46fed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:17.225894 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd79b9-01f8-4187-9e57-96f6ea46fed1 admin admin] PATH_INFO: `/v3/projects/444a8b826298487584ddaf3d2122b06a/users/8b4d9bbd0db54bb699cfd642c1a07242/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:17.227832 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd79b9-01f8-4187-9e57-96f6ea46fed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=444a8b826298487584ddaf3d2122b06a, user_id=8b4d9bbd0db54bb699cfd642c1a07242, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:17.230454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd79b9-01f8-4187-9e57-96f6ea46fed1 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:17.241098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1253/2515] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:17 2022] PUT /identity/v3/projects/444a8b826298487584ddaf3d2122b06a/users/8b4d9bbd0db54bb699cfd642c1a07242/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:05:17.252951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4730328-77a6-4cf4-8faf-beaa7e46ae43 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:17.253241 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4730328-77a6-4cf4-8faf-beaa7e46ae43 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:17.253518 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4730328-77a6-4cf4-8faf-beaa7e46ae43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:17.284514 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-b4730328-77a6-4cf4-8faf-beaa7e46ae43 None None] MFA Rules not processed for user `8b4d9bbd0db54bb699cfd642c1a07242`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:17.285933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b4730328-77a6-4cf4-8faf-beaa7e46ae43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:17.363908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1263/2516] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:17 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:17.380141 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd79b9-01f8-4187-9e57-96f6ea46fed1 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:17.382065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-04cd79b9-01f8-4187-9e57-96f6ea46fed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:17.564707 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:17.572108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd79b9-01f8-4187-9e57-96f6ea46fed1 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:17.577603 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e214d-0547-4f40-a7e2-6c34b1038ab1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:17.578228 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e214d-0547-4f40-a7e2-6c34b1038ab1 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:17.578505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e214d-0547-4f40-a7e2-6c34b1038ab1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:17.578807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e214d-0547-4f40-a7e2-6c34b1038ab1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:17.590364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e214d-0547-4f40-a7e2-6c34b1038ab1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:17.591868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e214d-0547-4f40-a7e2-6c34b1038ab1 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:17.603515 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1254/2517] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:17 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:18.037099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4730328-77a6-4cf4-8faf-beaa7e46ae43 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:18.110302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4730328-77a6-4cf4-8faf-beaa7e46ae43 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:18.114695 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f72e79-34f6-4ee5-b701-989aac52a444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:18.115358 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f72e79-34f6-4ee5-b701-989aac52a444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:18.115634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f72e79-34f6-4ee5-b701-989aac52a444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:18.115902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f72e79-34f6-4ee5-b701-989aac52a444 admin admin] PATH_INFO: `/v3/users/c4c567514ee5472b943d0f7ae35ed6cc` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:18.117027 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f72e79-34f6-4ee5-b701-989aac52a444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4c567514ee5472b943d0f7ae35ed6cc)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:18.119070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f72e79-34f6-4ee5-b701-989aac52a444 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:18.167784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-02f72e79-34f6-4ee5-b701-989aac52a444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4c567514ee5472b943d0f7ae35ed6cc'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:18.178009 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-02f72e79-34f6-4ee5-b701-989aac52a444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4c567514ee5472b943d0f7ae35ed6cc'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:18.186049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-02f72e79-34f6-4ee5-b701-989aac52a444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4c567514ee5472b943d0f7ae35ed6cc'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:18.194974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:18.201866 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-02f72e79-34f6-4ee5-b701-989aac52a444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4c567514ee5472b943d0f7ae35ed6cc'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:18.203105 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-02f72e79-34f6-4ee5-b701-989aac52a444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4c567514ee5472b943d0f7ae35ed6cc'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:18.210475 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1264/2518] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:05:18 2022] DELETE /identity/v3/users/c4c567514ee5472b943d0f7ae35ed6cc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:18.215698 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae96594-40a5-41e1-994a-30e69c68c97e None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:18.215979 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae96594-40a5-41e1-994a-30e69c68c97e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:18.216331 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae96594-40a5-41e1-994a-30e69c68c97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:18.240525 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-4ae96594-40a5-41e1-994a-30e69c68c97e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:18.241745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-4ae96594-40a5-41e1-994a-30e69c68c97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:18.299934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1255/2519] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:18.309799 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f72e79-34f6-4ee5-b701-989aac52a444 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:18.311315 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-02f72e79-34f6-4ee5-b701-989aac52a444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:18.368810 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f72e79-34f6-4ee5-b701-989aac52a444 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:18.372443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ec919-f189-4182-b3ae-2af5989dc2ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:18.373044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ec919-f189-4182-b3ae-2af5989dc2ca service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:18.373320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ec919-f189-4182-b3ae-2af5989dc2ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:18.373586 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ec919-f189-4182-b3ae-2af5989dc2ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:18.382538 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ec919-f189-4182-b3ae-2af5989dc2ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:18.383996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ec919-f189-4182-b3ae-2af5989dc2ca service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:18.393936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1265/2520] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:18.736763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae96594-40a5-41e1-994a-30e69c68c97e None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:18.738256 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-4ae96594-40a5-41e1-994a-30e69c68c97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:18.754127 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae96594-40a5-41e1-994a-30e69c68c97e None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:18.761237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39bdb7c-0638-47bf-b14b-72f6a84d074a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:18.762446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bdb7c-0638-47bf-b14b-72f6a84d074a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:18.762713 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bdb7c-0638-47bf-b14b-72f6a84d074a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:18.763206 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39bdb7c-0638-47bf-b14b-72f6a84d074a admin admin] PATH_INFO: `/v3/projects/b5213f7f369b41059977afd16a0eac64` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:18.764518 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39bdb7c-0638-47bf-b14b-72f6a84d074a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5213f7f369b41059977afd16a0eac64)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:18.766581 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39bdb7c-0638-47bf-b14b-72f6a84d074a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:18.820012 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9364c1f-016b-4731-828a-a7003338b9f2 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:18.820359 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9364c1f-016b-4731-828a-a7003338b9f2 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:18.820600 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9364c1f-016b-4731-828a-a7003338b9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:18.831026 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f39bdb7c-0638-47bf-b14b-72f6a84d074a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5213f7f369b41059977afd16a0eac64', 'request_id': 'req-f39bdb7c-0638-47bf-b14b-72f6a84d074a'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:18.839855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f39bdb7c-0638-47bf-b14b-72f6a84d074a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5213f7f369b41059977afd16a0eac64', 'request_id': 'req-f39bdb7c-0638-47bf-b14b-72f6a84d074a'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:18.844190 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-a9364c1f-016b-4731-828a-a7003338b9f2 None None] MFA Rules not processed for user `6256407d9df648d5bbe33495cb9a8b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:18.845512 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a9364c1f-016b-4731-828a-a7003338b9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:18.848404 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f39bdb7c-0638-47bf-b14b-72f6a84d074a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5213f7f369b41059977afd16a0eac64', 'request_id': 'req-f39bdb7c-0638-47bf-b14b-72f6a84d074a'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:18.861919 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f39bdb7c-0638-47bf-b14b-72f6a84d074a admin admin] The token cache is being invalidate because project b5213f7f369b41059977afd16a0eac64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:05:18.862337 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f39bdb7c-0638-47bf-b14b-72f6a84d074a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:18.863053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f39bdb7c-0638-47bf-b14b-72f6a84d074a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:18.864547 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1256/2521] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:05:18 2022] DELETE /identity/v3/projects/b5213f7f369b41059977afd16a0eac64 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:18.868595 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39bdb7c-0638-47bf-b14b-72f6a84d074a admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:18.870270 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-f39bdb7c-0638-47bf-b14b-72f6a84d074a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:18.919798 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1266/2522] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Sep 22 15:05:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:05:18.938458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9364c1f-016b-4731-828a-a7003338b9f2 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:18.941195 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a9364c1f-016b-4731-828a-a7003338b9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:18.943705 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39bdb7c-0638-47bf-b14b-72f6a84d074a admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:18.950055 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea832c2-3593-4c5c-b9ae-ac93b0aa25dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:18.950794 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea832c2-3593-4c5c-b9ae-ac93b0aa25dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:18.951021 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea832c2-3593-4c5c-b9ae-ac93b0aa25dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:18.951292 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea832c2-3593-4c5c-b9ae-ac93b0aa25dd admin admin] PATH_INFO: `/v3/users/8b4d9bbd0db54bb699cfd642c1a07242` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:18.952784 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea832c2-3593-4c5c-b9ae-ac93b0aa25dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b4d9bbd0db54bb699cfd642c1a07242)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:18.954323 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea832c2-3593-4c5c-b9ae-ac93b0aa25dd admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:18.968521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:19.011702 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2ea832c2-3593-4c5c-b9ae-ac93b0aa25dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b4d9bbd0db54bb699cfd642c1a07242'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:19.021596 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2ea832c2-3593-4c5c-b9ae-ac93b0aa25dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b4d9bbd0db54bb699cfd642c1a07242'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:19.022257 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2ea832c2-3593-4c5c-b9ae-ac93b0aa25dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b4d9bbd0db54bb699cfd642c1a07242'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:19.031561 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9364c1f-016b-4731-828a-a7003338b9f2 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:19.036817 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f189c9-f815-463c-8343-f0fcad993de2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a265623ae734936a340c97df852e227', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:19.037432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f189c9-f815-463c-8343-f0fcad993de2 service swift] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:19.038010 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f189c9-f815-463c-8343-f0fcad993de2 service swift] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:19.038118 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2ea832c2-3593-4c5c-b9ae-ac93b0aa25dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b4d9bbd0db54bb699cfd642c1a07242'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:19.038206 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f189c9-f815-463c-8343-f0fcad993de2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:19.038684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-2ea832c2-3593-4c5c-b9ae-ac93b0aa25dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b4d9bbd0db54bb699cfd642c1a07242'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:19.039820 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-04f189c9-f815-463c-8343-f0fcad993de2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:19.054360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f189c9-f815-463c-8343-f0fcad993de2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:19.056102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f189c9-f815-463c-8343-f0fcad993de2 service swift] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:19.057234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1257/2523] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:05:18 2022] DELETE /identity/v3/users/8b4d9bbd0db54bb699cfd642c1a07242 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:19.066765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1267/2524] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Sep 22 15:05:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 22 15:05:19.498435 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea832c2-3593-4c5c-b9ae-ac93b0aa25dd admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:19.501271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-2ea832c2-3593-4c5c-b9ae-ac93b0aa25dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:19.629387 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea832c2-3593-4c5c-b9ae-ac93b0aa25dd admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:19.635093 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e51e47-e722-4a01-b7fd-063fb07c1fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:19.636002 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e51e47-e722-4a01-b7fd-063fb07c1fd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:19.636496 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e51e47-e722-4a01-b7fd-063fb07c1fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:19.636897 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e51e47-e722-4a01-b7fd-063fb07c1fd4 admin admin] PATH_INFO: `/v3/projects/444a8b826298487584ddaf3d2122b06a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:19.638487 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e51e47-e722-4a01-b7fd-063fb07c1fd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=444a8b826298487584ddaf3d2122b06a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:19.640810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e51e47-e722-4a01-b7fd-063fb07c1fd4 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:19.712291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f7e51e47-e722-4a01-b7fd-063fb07c1fd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '444a8b826298487584ddaf3d2122b06a', 'request_id': 'req-f7e51e47-e722-4a01-b7fd-063fb07c1fd4'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:19.724346 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f7e51e47-e722-4a01-b7fd-063fb07c1fd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '444a8b826298487584ddaf3d2122b06a', 'request_id': 'req-f7e51e47-e722-4a01-b7fd-063fb07c1fd4'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:19.735995 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f7e51e47-e722-4a01-b7fd-063fb07c1fd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '444a8b826298487584ddaf3d2122b06a', 'request_id': 'req-f7e51e47-e722-4a01-b7fd-063fb07c1fd4'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:19.754614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f7e51e47-e722-4a01-b7fd-063fb07c1fd4 admin admin] The token cache is being invalidate because project 444a8b826298487584ddaf3d2122b06a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:05:19.755401 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f7e51e47-e722-4a01-b7fd-063fb07c1fd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:19.755978 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f7e51e47-e722-4a01-b7fd-063fb07c1fd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:19.758857 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1258/2525] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:05:19 2022] DELETE /identity/v3/projects/444a8b826298487584ddaf3d2122b06a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:19.764657 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d54968-d40c-46ab-b128-a609c463a503 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:19.765066 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d54968-d40c-46ab-b128-a609c463a503 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:19.765444 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d54968-d40c-46ab-b128-a609c463a503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:19.793729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-81d54968-d40c-46ab-b128-a609c463a503 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:19.796200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-81d54968-d40c-46ab-b128-a609c463a503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:19.867240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:19.940628 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1268/2526] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:19.968973 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf484e-cfd9-44c5-b168-31408f036a10 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:19.969189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf484e-cfd9-44c5-b168-31408f036a10 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:19.969420 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf484e-cfd9-44c5-b168-31408f036a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:20.003852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-4ccf484e-cfd9-44c5-b168-31408f036a10 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:20.005096 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-4ccf484e-cfd9-44c5-b168-31408f036a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:20.029261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1259/2527] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:20.033534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d54968-d40c-46ab-b128-a609c463a503 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:20.046612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d54968-d40c-46ab-b128-a609c463a503 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:20.051854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9889405-5481-4040-8864-b16c896a3041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:20.059207 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9889405-5481-4040-8864-b16c896a3041 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:20.060049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9889405-5481-4040-8864-b16c896a3041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:20.060642 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9889405-5481-4040-8864-b16c896a3041 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:20.061724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9889405-5481-4040-8864-b16c896a3041 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:05:20.062362 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9889405-5481-4040-8864-b16c896a3041 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:20.064506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9889405-5481-4040-8864-b16c896a3041 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:20.091745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1269/2528] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:05:20.096168 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf484e-cfd9-44c5-b168-31408f036a10 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:20.106065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf484e-cfd9-44c5-b168-31408f036a10 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:20.111298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273ac25-65c1-442b-acd6-076d914addf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:20.111910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273ac25-65c1-442b-acd6-076d914addf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:20.112187 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273ac25-65c1-442b-acd6-076d914addf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:20.112436 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d273ac25-65c1-442b-acd6-076d914addf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:20.113131 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273ac25-65c1-442b-acd6-076d914addf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:20.115172 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d273ac25-65c1-442b-acd6-076d914addf8 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:20.137246 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:20.154469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1260/2529] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:05:20 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:20.160134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9889405-5481-4040-8864-b16c896a3041 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:20.172331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9889405-5481-4040-8864-b16c896a3041 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:20.250743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f652e24-eb09-4b2b-a795-dffb87367700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:20.251457 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f652e24-eb09-4b2b-a795-dffb87367700 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:20.251827 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f652e24-eb09-4b2b-a795-dffb87367700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:20.252105 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f652e24-eb09-4b2b-a795-dffb87367700 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:20.253068 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f652e24-eb09-4b2b-a795-dffb87367700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:20.255266 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f652e24-eb09-4b2b-a795-dffb87367700 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:20.303602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1270/2530] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:05:20 2022] POST /identity/v3/users => generated 288 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:20.308242 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273ac25-65c1-442b-acd6-076d914addf8 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:20.319628 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d273ac25-65c1-442b-acd6-076d914addf8 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:20.339044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fd783-1b47-415d-a91a-6c1794594709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:20.339704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fd783-1b47-415d-a91a-6c1794594709 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:20.340572 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fd783-1b47-415d-a91a-6c1794594709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:20.340887 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fd783-1b47-415d-a91a-6c1794594709 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:20.341611 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fd783-1b47-415d-a91a-6c1794594709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:20.343313 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fd783-1b47-415d-a91a-6c1794594709 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:20.364187 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1261/2531] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:20.368484 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f652e24-eb09-4b2b-a795-dffb87367700 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:20.382533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f652e24-eb09-4b2b-a795-dffb87367700 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:20.388106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92512c35-b56b-4977-b15d-f30bd1a878f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:20.388819 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92512c35-b56b-4977-b15d-f30bd1a878f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:20.389113 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92512c35-b56b-4977-b15d-f30bd1a878f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:20.389506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92512c35-b56b-4977-b15d-f30bd1a878f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:20.390253 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92512c35-b56b-4977-b15d-f30bd1a878f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:20.392317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92512c35-b56b-4977-b15d-f30bd1a878f5 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:20.414668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fd783-1b47-415d-a91a-6c1794594709 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:20.421803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1271/2532] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:05:20.436254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fd783-1b47-415d-a91a-6c1794594709 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:20.467548 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7eff2-de0f-494c-9832-05a802e9973a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:20.468242 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7eff2-de0f-494c-9832-05a802e9973a admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:20.468523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7eff2-de0f-494c-9832-05a802e9973a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:20.468801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7eff2-de0f-494c-9832-05a802e9973a admin admin] PATH_INFO: `/v3/projects/50d468dd786549b0b4394a6115d1a958/users/32926dcca0244286af6590388bb98e73/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:20.493145 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f7eff2-de0f-494c-9832-05a802e9973a admin admin] RBAC: Authorizing `identity:create_grant(project_id=50d468dd786549b0b4394a6115d1a958, user_id=32926dcca0244286af6590388bb98e73, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:20.501363 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f7eff2-de0f-494c-9832-05a802e9973a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:20.546909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1262/2533] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:20 2022] PUT /identity/v3/projects/50d468dd786549b0b4394a6115d1a958/users/32926dcca0244286af6590388bb98e73/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:20.553564 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4790ea-9800-4c26-8b22-677383b856b4 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:20.554033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4790ea-9800-4c26-8b22-677383b856b4 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:20.554420 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4790ea-9800-4c26-8b22-677383b856b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:20.607847 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-af4790ea-9800-4c26-8b22-677383b856b4 None None] MFA Rules not processed for user `32926dcca0244286af6590388bb98e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:20.615945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-af4790ea-9800-4c26-8b22-677383b856b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:20.741250 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1272/2534] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:20.745713 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7eff2-de0f-494c-9832-05a802e9973a admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:20.814264 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7eff2-de0f-494c-9832-05a802e9973a admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:20.820832 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e6cde-e9fd-43d9-9fa7-addc80306198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:20.821775 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e6cde-e9fd-43d9-9fa7-addc80306198 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:20.822340 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e6cde-e9fd-43d9-9fa7-addc80306198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:20.822915 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e6cde-e9fd-43d9-9fa7-addc80306198 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:20.823870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e6cde-e9fd-43d9-9fa7-addc80306198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:20.826872 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e6cde-e9fd-43d9-9fa7-addc80306198 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:20.860579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1263/2535] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:05:20 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:20.864755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4790ea-9800-4c26-8b22-677383b856b4 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:20.876618 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4790ea-9800-4c26-8b22-677383b856b4 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:20.897125 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:20.943508 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0a590-cc7a-4b78-a563-6ee5fd36a620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:20.944317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0a590-cc7a-4b78-a563-6ee5fd36a620 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:20.944729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0a590-cc7a-4b78-a563-6ee5fd36a620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:20.945129 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0a590-cc7a-4b78-a563-6ee5fd36a620 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:20.945947 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0a590-cc7a-4b78-a563-6ee5fd36a620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:20.948095 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0a590-cc7a-4b78-a563-6ee5fd36a620 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:20.979024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1273/2536] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:05:20 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:20.983786 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e6cde-e9fd-43d9-9fa7-addc80306198 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:21.001508 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e6cde-e9fd-43d9-9fa7-addc80306198 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:21.015933 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e82df-d79d-4033-a034-7a0f5bdd263d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:21.016655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e82df-d79d-4033-a034-7a0f5bdd263d admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:21.017004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e82df-d79d-4033-a034-7a0f5bdd263d admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:21.017330 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e82df-d79d-4033-a034-7a0f5bdd263d admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:21.018495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e82df-d79d-4033-a034-7a0f5bdd263d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:21.019916 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e82df-d79d-4033-a034-7a0f5bdd263d admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:21.031631 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1264/2537] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:21.035932 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0a590-cc7a-4b78-a563-6ee5fd36a620 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:21.046189 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0a590-cc7a-4b78-a563-6ee5fd36a620 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:21.051108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e698d-b9d1-4313-baba-a505088c7ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:21.051770 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e698d-b9d1-4313-baba-a505088c7ec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:21.052047 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e698d-b9d1-4313-baba-a505088c7ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:21.052317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e698d-b9d1-4313-baba-a505088c7ec0 admin admin] PATH_INFO: `/v3/projects/0d1fcf8ae58b4dc6b4319eeab2c31aea/users/bdcef9816277447384d36092647b1bd1/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:21.070357 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e698d-b9d1-4313-baba-a505088c7ec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d1fcf8ae58b4dc6b4319eeab2c31aea, user_id=bdcef9816277447384d36092647b1bd1, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:21.073139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e698d-b9d1-4313-baba-a505088c7ec0 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:21.094862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1274/2538] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:21 2022] PUT /identity/v3/projects/0d1fcf8ae58b4dc6b4319eeab2c31aea/users/bdcef9816277447384d36092647b1bd1/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:21.095940 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e82df-d79d-4033-a034-7a0f5bdd263d admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:21.107074 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e82df-d79d-4033-a034-7a0f5bdd263d admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:21.185138 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e6336-5dce-4f0a-9db4-ca32d61267b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:21.185868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e6336-5dce-4f0a-9db4-ca32d61267b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:21.186195 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e6336-5dce-4f0a-9db4-ca32d61267b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:21.186540 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e6336-5dce-4f0a-9db4-ca32d61267b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:21.187295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e6336-5dce-4f0a-9db4-ca32d61267b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:21.188955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e6336-5dce-4f0a-9db4-ca32d61267b0 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:21.200710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1265/2539] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:05:21.205623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e698d-b9d1-4313-baba-a505088c7ec0 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:21.217801 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e698d-b9d1-4313-baba-a505088c7ec0 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:21.222631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14965955-80a7-4b3b-8ce8-04f0d30e0089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:21.223403 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14965955-80a7-4b3b-8ce8-04f0d30e0089 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:21.223758 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14965955-80a7-4b3b-8ce8-04f0d30e0089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:21.224262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14965955-80a7-4b3b-8ce8-04f0d30e0089 admin admin] PATH_INFO: `/v3/projects/0d1fcf8ae58b4dc6b4319eeab2c31aea/users/bdcef9816277447384d36092647b1bd1/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:21.226328 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14965955-80a7-4b3b-8ce8-04f0d30e0089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d1fcf8ae58b4dc6b4319eeab2c31aea, user_id=bdcef9816277447384d36092647b1bd1, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:21.229155 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14965955-80a7-4b3b-8ce8-04f0d30e0089 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:21.240992 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1275/2540] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:21 2022] PUT /identity/v3/projects/0d1fcf8ae58b4dc6b4319eeab2c31aea/users/bdcef9816277447384d36092647b1bd1/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:21.252191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549f3de-cc8d-459b-a0d9-125247c5b278 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:21.252392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549f3de-cc8d-459b-a0d9-125247c5b278 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:21.252550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549f3de-cc8d-459b-a0d9-125247c5b278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:21.283345 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-d549f3de-cc8d-459b-a0d9-125247c5b278 None None] MFA Rules not processed for user `bdcef9816277447384d36092647b1bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:21.287560 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d549f3de-cc8d-459b-a0d9-125247c5b278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:21.312663 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:21.381626 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1266/2541] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:21.392642 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14965955-80a7-4b3b-8ce8-04f0d30e0089 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:21.394437 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-14965955-80a7-4b3b-8ce8-04f0d30e0089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:21.457032 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14965955-80a7-4b3b-8ce8-04f0d30e0089 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:21.460930 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac1b0b-eb6d-422c-9999-718f9cb01e44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:21.461537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac1b0b-eb6d-422c-9999-718f9cb01e44 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:21.461818 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac1b0b-eb6d-422c-9999-718f9cb01e44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:21.462089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac1b0b-eb6d-422c-9999-718f9cb01e44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:21.535645 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac1b0b-eb6d-422c-9999-718f9cb01e44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:21.537326 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac1b0b-eb6d-422c-9999-718f9cb01e44 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:21.555436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1276/2542] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:24.273252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549f3de-cc8d-459b-a0d9-125247c5b278 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:24.361814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549f3de-cc8d-459b-a0d9-125247c5b278 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:24.367283 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba56af4-dc05-4349-9331-e8bfc4e17406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:24.368014 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba56af4-dc05-4349-9331-e8bfc4e17406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:24.368351 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba56af4-dc05-4349-9331-e8bfc4e17406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:24.368627 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba56af4-dc05-4349-9331-e8bfc4e17406 admin admin] PATH_INFO: `/v3/users/32926dcca0244286af6590388bb98e73` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:24.370045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba56af4-dc05-4349-9331-e8bfc4e17406 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32926dcca0244286af6590388bb98e73)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:24.372799 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba56af4-dc05-4349-9331-e8bfc4e17406 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:24.439473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-aba56af4-dc05-4349-9331-e8bfc4e17406 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32926dcca0244286af6590388bb98e73'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:24.451970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-aba56af4-dc05-4349-9331-e8bfc4e17406 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32926dcca0244286af6590388bb98e73'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:24.453113 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-aba56af4-dc05-4349-9331-e8bfc4e17406 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32926dcca0244286af6590388bb98e73'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:24.473169 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-aba56af4-dc05-4349-9331-e8bfc4e17406 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32926dcca0244286af6590388bb98e73'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:24.475227 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-aba56af4-dc05-4349-9331-e8bfc4e17406 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32926dcca0244286af6590388bb98e73'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:24.490786 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1267/2543] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:05:24 2022] DELETE /identity/v3/users/32926dcca0244286af6590388bb98e73 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:24.495100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b13c87-f475-45cf-b836-9bf4328a4bc8 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:24.495612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b13c87-f475-45cf-b836-9bf4328a4bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:24.495707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b13c87-f475-45cf-b836-9bf4328a4bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:24.516956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-92b13c87-f475-45cf-b836-9bf4328a4bc8 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:24.518194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-92b13c87-f475-45cf-b836-9bf4328a4bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:24.554650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:24.639141 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1277/2544] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:24.648939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba56af4-dc05-4349-9331-e8bfc4e17406 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:24.650412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-aba56af4-dc05-4349-9331-e8bfc4e17406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:24.730184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:24.774723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba56af4-dc05-4349-9331-e8bfc4e17406 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:24.779054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28679e-7adf-4a7f-9a6a-d07bcb002572 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:24.779669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28679e-7adf-4a7f-9a6a-d07bcb002572 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:24.779943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28679e-7adf-4a7f-9a6a-d07bcb002572 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:24.780209 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28679e-7adf-4a7f-9a6a-d07bcb002572 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:24.793470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28679e-7adf-4a7f-9a6a-d07bcb002572 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:24.797336 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28679e-7adf-4a7f-9a6a-d07bcb002572 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:24.814301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1268/2545] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:25.162197 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b13c87-f475-45cf-b836-9bf4328a4bc8 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:25.163700 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-92b13c87-f475-45cf-b836-9bf4328a4bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:25.177531 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b13c87-f475-45cf-b836-9bf4328a4bc8 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:25.183120 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:25.184125 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:25.184669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:25.195033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48 admin admin] PATH_INFO: `/v3/projects/50d468dd786549b0b4394a6115d1a958` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:25.196583 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50d468dd786549b0b4394a6115d1a958)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:25.198627 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:25.257727 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50d468dd786549b0b4394a6115d1a958', 'request_id': 'req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:25.264655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50d468dd786549b0b4394a6115d1a958', 'request_id': 'req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:25.270638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50d468dd786549b0b4394a6115d1a958', 'request_id': 'req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:25.286145 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48 admin admin] The token cache is being invalidate because project 50d468dd786549b0b4394a6115d1a958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:05:25.286504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:25.287104 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:25.292650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1278/2546] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:05:25 2022] DELETE /identity/v3/projects/50d468dd786549b0b4394a6115d1a958 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:25.294059 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28679e-7adf-4a7f-9a6a-d07bcb002572 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:25.304092 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28679e-7adf-4a7f-9a6a-d07bcb002572 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:25.374924 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7f190-a823-4206-b48b-88aac541ac5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:25.375595 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7f190-a823-4206-b48b-88aac541ac5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:25.375875 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7f190-a823-4206-b48b-88aac541ac5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:25.376145 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7f190-a823-4206-b48b-88aac541ac5b admin admin] PATH_INFO: `/v3/users/bdcef9816277447384d36092647b1bd1` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:25.377378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7f190-a823-4206-b48b-88aac541ac5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdcef9816277447384d36092647b1bd1)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:25.379694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7f190-a823-4206-b48b-88aac541ac5b admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:25.441342 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f0e7f190-a823-4206-b48b-88aac541ac5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdcef9816277447384d36092647b1bd1'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:25.450904 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f0e7f190-a823-4206-b48b-88aac541ac5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdcef9816277447384d36092647b1bd1'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:25.452070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f0e7f190-a823-4206-b48b-88aac541ac5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdcef9816277447384d36092647b1bd1'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:25.469200 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f0e7f190-a823-4206-b48b-88aac541ac5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdcef9816277447384d36092647b1bd1'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:25.469955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f0e7f190-a823-4206-b48b-88aac541ac5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdcef9816277447384d36092647b1bd1'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:25.484130 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1269/2547] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:05:25 2022] DELETE /identity/v3/users/bdcef9816277447384d36092647b1bd1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:25.869548 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:25.871566 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:25.907740 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de834af-7d9c-49bd-bfc8-3906a4c0ac48 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:26.001005 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b926f5-0430-4628-9f08-a22ea53dab09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:26.002654 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b926f5-0430-4628-9f08-a22ea53dab09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:26.003038 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b926f5-0430-4628-9f08-a22ea53dab09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:26.003388 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b926f5-0430-4628-9f08-a22ea53dab09 admin admin] PATH_INFO: `/v3/projects/0d1fcf8ae58b4dc6b4319eeab2c31aea` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:26.004649 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b926f5-0430-4628-9f08-a22ea53dab09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d1fcf8ae58b4dc6b4319eeab2c31aea)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:26.006823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b926f5-0430-4628-9f08-a22ea53dab09 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:26.073194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b4b926f5-0430-4628-9f08-a22ea53dab09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d1fcf8ae58b4dc6b4319eeab2c31aea', 'request_id': 'req-b4b926f5-0430-4628-9f08-a22ea53dab09'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:26.090498 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b4b926f5-0430-4628-9f08-a22ea53dab09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d1fcf8ae58b4dc6b4319eeab2c31aea', 'request_id': 'req-b4b926f5-0430-4628-9f08-a22ea53dab09'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:26.093239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:26.101282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b4b926f5-0430-4628-9f08-a22ea53dab09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d1fcf8ae58b4dc6b4319eeab2c31aea', 'request_id': 'req-b4b926f5-0430-4628-9f08-a22ea53dab09'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:26.117920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b4b926f5-0430-4628-9f08-a22ea53dab09 admin admin] The token cache is being invalidate because project 0d1fcf8ae58b4dc6b4319eeab2c31aea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:05:26.118270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b4b926f5-0430-4628-9f08-a22ea53dab09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:26.118894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b4b926f5-0430-4628-9f08-a22ea53dab09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:26.120681 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1279/2548] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:05:25 2022] DELETE /identity/v3/projects/0d1fcf8ae58b4dc6b4319eeab2c31aea => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:26.125531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13d1db-3675-4686-b0c9-83827eb150a3 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:26.125877 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13d1db-3675-4686-b0c9-83827eb150a3 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:26.126077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13d1db-3675-4686-b0c9-83827eb150a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:26.147428 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-0f13d1db-3675-4686-b0c9-83827eb150a3 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:26.148662 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-0f13d1db-3675-4686-b0c9-83827eb150a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:26.200061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:26.214143 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1270/2549] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:26.279271 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a1007-728c-4365-ab24-312a2fde6a2e None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:26.279639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a1007-728c-4365-ab24-312a2fde6a2e None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:26.279840 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a1007-728c-4365-ab24-312a2fde6a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:26.302899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-248a1007-728c-4365-ab24-312a2fde6a2e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:26.304146 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-248a1007-728c-4365-ab24-312a2fde6a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:26.320000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1280/2550] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:26.324395 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13d1db-3675-4686-b0c9-83827eb150a3 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:26.335467 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13d1db-3675-4686-b0c9-83827eb150a3 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:26.340150 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2910e10-953e-4244-867b-c5d7e93096a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:26.341370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2910e10-953e-4244-867b-c5d7e93096a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:26.341598 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2910e10-953e-4244-867b-c5d7e93096a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:26.341947 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2910e10-953e-4244-867b-c5d7e93096a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:26.342487 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2910e10-953e-4244-867b-c5d7e93096a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:05:26.342829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2910e10-953e-4244-867b-c5d7e93096a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:26.344421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2910e10-953e-4244-867b-c5d7e93096a0 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:26.357639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1271/2551] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:05:26.362897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a1007-728c-4365-ab24-312a2fde6a2e None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:26.374769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a1007-728c-4365-ab24-312a2fde6a2e None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:26.379650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0d426-de62-42bc-9376-adc475697424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:26.380260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0d426-de62-42bc-9376-adc475697424 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:26.380534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0d426-de62-42bc-9376-adc475697424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:26.380801 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0d426-de62-42bc-9376-adc475697424 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:26.381439 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0d426-de62-42bc-9376-adc475697424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:26.383561 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0d426-de62-42bc-9376-adc475697424 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:26.414240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1281/2552] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:05:26 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:26.418435 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2910e10-953e-4244-867b-c5d7e93096a0 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:26.429316 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2910e10-953e-4244-867b-c5d7e93096a0 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:26.496503 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69acd6-4041-4a18-aa59-04686fd9925b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:26.497341 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69acd6-4041-4a18-aa59-04686fd9925b admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:26.497649 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69acd6-4041-4a18-aa59-04686fd9925b admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:26.497931 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69acd6-4041-4a18-aa59-04686fd9925b admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:26.498653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69acd6-4041-4a18-aa59-04686fd9925b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:26.500938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69acd6-4041-4a18-aa59-04686fd9925b admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:26.533468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0d426-de62-42bc-9376-adc475697424 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:26.538172 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1272/2553] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:05:26 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:05:26.545621 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0d426-de62-42bc-9376-adc475697424 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:26.551324 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0c3a1-6523-4706-9551-d5a780fa1378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:26.551941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0c3a1-6523-4706-9551-d5a780fa1378 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:26.552219 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0c3a1-6523-4706-9551-d5a780fa1378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:26.552488 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0c3a1-6523-4706-9551-d5a780fa1378 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:26.553139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0c3a1-6523-4706-9551-d5a780fa1378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:26.554787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0c3a1-6523-4706-9551-d5a780fa1378 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:26.571553 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1282/2554] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:26.572743 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69acd6-4041-4a18-aa59-04686fd9925b admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:26.585517 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69acd6-4041-4a18-aa59-04686fd9925b admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:26.595711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff50cae-2d3b-4428-a44f-0bbccc40b8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:26.596985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff50cae-2d3b-4428-a44f-0bbccc40b8c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:26.598033 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff50cae-2d3b-4428-a44f-0bbccc40b8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:26.598433 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff50cae-2d3b-4428-a44f-0bbccc40b8c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:26.599521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff50cae-2d3b-4428-a44f-0bbccc40b8c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:26.601973 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff50cae-2d3b-4428-a44f-0bbccc40b8c0 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:26.621766 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1273/2555] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:26.624584 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0c3a1-6523-4706-9551-d5a780fa1378 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:26.635970 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0c3a1-6523-4706-9551-d5a780fa1378 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:26.641494 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d41785-5880-44f7-beb7-9500b7b614f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:26.642257 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d41785-5880-44f7-beb7-9500b7b614f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:26.642610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d41785-5880-44f7-beb7-9500b7b614f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:26.642999 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d41785-5880-44f7-beb7-9500b7b614f8 admin admin] PATH_INFO: `/v3/projects/ab6ca70d427646d3b0a8ebd2838c807b/users/b761e143a6264e66aac0caac8cb46135/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:26.661965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d41785-5880-44f7-beb7-9500b7b614f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab6ca70d427646d3b0a8ebd2838c807b, user_id=b761e143a6264e66aac0caac8cb46135, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:26.665398 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d41785-5880-44f7-beb7-9500b7b614f8 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:26.679595 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1283/2556] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:26 2022] PUT /identity/v3/projects/ab6ca70d427646d3b0a8ebd2838c807b/users/b761e143a6264e66aac0caac8cb46135/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:26.684692 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea05652-66df-4769-b068-df76171b271d None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:26.684970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea05652-66df-4769-b068-df76171b271d None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:26.685245 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea05652-66df-4769-b068-df76171b271d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:26.707574 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-9ea05652-66df-4769-b068-df76171b271d None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:26.709038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-9ea05652-66df-4769-b068-df76171b271d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:26.770925 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1274/2557] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:26.781854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d41785-5880-44f7-beb7-9500b7b614f8 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:26.783655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-49d41785-5880-44f7-beb7-9500b7b614f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:26.864112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d41785-5880-44f7-beb7-9500b7b614f8 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:26.868321 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afe736-0fc8-4cc3-a627-db349ead8ce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:26.869233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afe736-0fc8-4cc3-a627-db349ead8ce0 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:26.869612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afe736-0fc8-4cc3-a627-db349ead8ce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:26.870207 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afe736-0fc8-4cc3-a627-db349ead8ce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:26.881317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2afe736-0fc8-4cc3-a627-db349ead8ce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:26.883136 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2afe736-0fc8-4cc3-a627-db349ead8ce0 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:26.895970 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1284/2558] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:32.515402 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77fd1c-a0c5-424e-ba38-2c9f70bcdc4b None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:32.516013 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77fd1c-a0c5-424e-ba38-2c9f70bcdc4b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:32.516013 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77fd1c-a0c5-424e-ba38-2c9f70bcdc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:32.578683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-3e77fd1c-a0c5-424e-ba38-2c9f70bcdc4b None None] MFA Rules not processed for user `b761e143a6264e66aac0caac8cb46135`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:32.580535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3e77fd1c-a0c5-424e-ba38-2c9f70bcdc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:32.662682 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:32.692394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1275/2559] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:32.696653 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afe736-0fc8-4cc3-a627-db349ead8ce0 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:32.710213 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afe736-0fc8-4cc3-a627-db349ead8ce0 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:32.715559 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627ced3-ebc9-4a3c-8646-f2f41d69ae22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:32.716174 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627ced3-ebc9-4a3c-8646-f2f41d69ae22 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:32.716451 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627ced3-ebc9-4a3c-8646-f2f41d69ae22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:32.716723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627ced3-ebc9-4a3c-8646-f2f41d69ae22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:32.717363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627ced3-ebc9-4a3c-8646-f2f41d69ae22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:32.719507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627ced3-ebc9-4a3c-8646-f2f41d69ae22 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:32.772776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1285/2560] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:05:32 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:32.785124 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77fd1c-a0c5-424e-ba38-2c9f70bcdc4b None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:32.801706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77fd1c-a0c5-424e-ba38-2c9f70bcdc4b None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:32.902182 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452e50e-9f74-4482-bc96-464dda16cc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:32.903420 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452e50e-9f74-4482-bc96-464dda16cc28 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:32.903703 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452e50e-9f74-4482-bc96-464dda16cc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:32.903979 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452e50e-9f74-4482-bc96-464dda16cc28 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:32.905263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452e50e-9f74-4482-bc96-464dda16cc28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:32.907274 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452e50e-9f74-4482-bc96-464dda16cc28 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:32.944591 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1276/2561] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:05:32 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:32.948495 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627ced3-ebc9-4a3c-8646-f2f41d69ae22 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:32.979224 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627ced3-ebc9-4a3c-8646-f2f41d69ae22 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:33.013883 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb7c08-42cf-494e-9c68-9363ec034abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:33.014602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb7c08-42cf-494e-9c68-9363ec034abb admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:33.014971 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb7c08-42cf-494e-9c68-9363ec034abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:33.015307 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb7c08-42cf-494e-9c68-9363ec034abb admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:33.016025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb7c08-42cf-494e-9c68-9363ec034abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:33.017712 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb7c08-42cf-494e-9c68-9363ec034abb admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:33.032648 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1286/2562] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:05:33.036892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452e50e-9f74-4482-bc96-464dda16cc28 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:33.048578 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452e50e-9f74-4482-bc96-464dda16cc28 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:33.055075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8c007-79e7-44e3-9416-51df1eac3d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:33.055772 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8c007-79e7-44e3-9416-51df1eac3d79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:33.056053 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8c007-79e7-44e3-9416-51df1eac3d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:33.056327 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8c007-79e7-44e3-9416-51df1eac3d79 admin admin] PATH_INFO: `/v3/projects/9f145f38cb0b472186ee1ea62ccf702b/users/834183ae750241d7a1e4c4bc3d50f572/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:33.075421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8c007-79e7-44e3-9416-51df1eac3d79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f145f38cb0b472186ee1ea62ccf702b, user_id=834183ae750241d7a1e4c4bc3d50f572, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:33.078553 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8c007-79e7-44e3-9416-51df1eac3d79 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:33.094595 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1277/2563] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:33 2022] PUT /identity/v3/projects/9f145f38cb0b472186ee1ea62ccf702b/users/834183ae750241d7a1e4c4bc3d50f572/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:05:33.099365 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb7c08-42cf-494e-9c68-9363ec034abb admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:33.111966 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb7c08-42cf-494e-9c68-9363ec034abb admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:33.177115 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:33.191587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bdc5c-166f-4a81-aa91-56ca770eabdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:33.192207 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bdc5c-166f-4a81-aa91-56ca770eabdd admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:33.192611 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bdc5c-166f-4a81-aa91-56ca770eabdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:33.192887 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bdc5c-166f-4a81-aa91-56ca770eabdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:33.193532 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78bdc5c-166f-4a81-aa91-56ca770eabdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:33.195257 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78bdc5c-166f-4a81-aa91-56ca770eabdd admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:33.207299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1287/2564] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:33.219905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8c007-79e7-44e3-9416-51df1eac3d79 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:33.234668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8c007-79e7-44e3-9416-51df1eac3d79 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:33.240966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241360bc-0e3e-4a3c-8ae0-b2f01963b738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:33.241691 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241360bc-0e3e-4a3c-8ae0-b2f01963b738 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:33.241911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241360bc-0e3e-4a3c-8ae0-b2f01963b738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:33.243309 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241360bc-0e3e-4a3c-8ae0-b2f01963b738 admin admin] PATH_INFO: `/v3/projects/9f145f38cb0b472186ee1ea62ccf702b/users/834183ae750241d7a1e4c4bc3d50f572/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:33.245946 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241360bc-0e3e-4a3c-8ae0-b2f01963b738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f145f38cb0b472186ee1ea62ccf702b, user_id=834183ae750241d7a1e4c4bc3d50f572, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:33.250198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241360bc-0e3e-4a3c-8ae0-b2f01963b738 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:33.268666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1278/2565] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:33 2022] PUT /identity/v3/projects/9f145f38cb0b472186ee1ea62ccf702b/users/834183ae750241d7a1e4c4bc3d50f572/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:37.777959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bdc5c-166f-4a81-aa91-56ca770eabdd admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:37.779442 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f78bdc5c-166f-4a81-aa91-56ca770eabdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:37.939610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bdc5c-166f-4a81-aa91-56ca770eabdd admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:38.060556 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5c18d-57da-437d-aec9-7e25e4c621da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:38.061484 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5c18d-57da-437d-aec9-7e25e4c621da service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:38.062032 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5c18d-57da-437d-aec9-7e25e4c621da service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:38.062487 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5c18d-57da-437d-aec9-7e25e4c621da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:38.064687 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-8ff5c18d-57da-437d-aec9-7e25e4c621da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:38.245074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5c18d-57da-437d-aec9-7e25e4c621da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:38.246620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5c18d-57da-437d-aec9-7e25e4c621da service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:38.259076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1288/2566] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:40.159528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccbd01-5b17-4d56-9375-b495e6fa1ddd None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:40.160102 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccbd01-5b17-4d56-9375-b495e6fa1ddd None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:40.160102 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eccbd01-5b17-4d56-9375-b495e6fa1ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:40.216340 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-7eccbd01-5b17-4d56-9375-b495e6fa1ddd None None] MFA Rules not processed for user `834183ae750241d7a1e4c4bc3d50f572`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:40.217608 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7eccbd01-5b17-4d56-9375-b495e6fa1ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:40.378959 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1279/2567] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:40.393586 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5c18d-57da-437d-aec9-7e25e4c621da service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:40.395205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-8ff5c18d-57da-437d-aec9-7e25e4c621da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:40.522882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=100 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:40.566515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5c18d-57da-437d-aec9-7e25e4c621da service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:40.573319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e03cb7-540c-4dc4-adc9-82ba20d6f2c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:40.574230 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e03cb7-540c-4dc4-adc9-82ba20d6f2c9 service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:40.575198 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e03cb7-540c-4dc4-adc9-82ba20d6f2c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:40.575626 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e03cb7-540c-4dc4-adc9-82ba20d6f2c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:40.670709 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e03cb7-540c-4dc4-adc9-82ba20d6f2c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:40.672433 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e03cb7-540c-4dc4-adc9-82ba20d6f2c9 service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:40.690581 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1289/2568] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:42.850680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b8591-1abb-4953-81b4-ebb34b32c6af None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:42.851302 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b8591-1abb-4953-81b4-ebb34b32c6af None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:42.851519 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b8591-1abb-4953-81b4-ebb34b32c6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:42.893082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-022b8591-1abb-4953-81b4-ebb34b32c6af None None] MFA Rules not processed for user `b761e143a6264e66aac0caac8cb46135`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:42.894328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-022b8591-1abb-4953-81b4-ebb34b32c6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:42.914490 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1280/2569] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:44.410607 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e03cb7-540c-4dc4-adc9-82ba20d6f2c9 service nova] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:44.412549 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-89e03cb7-540c-4dc4-adc9-82ba20d6f2c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:44.444029 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e03cb7-540c-4dc4-adc9-82ba20d6f2c9 service nova] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:44.449960 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5bd6f-6679-449b-8c8b-f7af5fd9f2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:44.451621 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5bd6f-6679-449b-8c8b-f7af5fd9f2e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:44.452029 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5bd6f-6679-449b-8c8b-f7af5fd9f2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:44.452636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5bd6f-6679-449b-8c8b-f7af5fd9f2e4 admin admin] PATH_INFO: `/v3/users/23d683cdc90041b5b29767fa2e2107e2` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:44.453933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5bd6f-6679-449b-8c8b-f7af5fd9f2e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23d683cdc90041b5b29767fa2e2107e2)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:44.456086 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5bd6f-6679-449b-8c8b-f7af5fd9f2e4 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:44.528162 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-31d5bd6f-6679-449b-8c8b-f7af5fd9f2e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23d683cdc90041b5b29767fa2e2107e2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:44.536922 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-31d5bd6f-6679-449b-8c8b-f7af5fd9f2e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23d683cdc90041b5b29767fa2e2107e2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:44.547220 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-31d5bd6f-6679-449b-8c8b-f7af5fd9f2e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23d683cdc90041b5b29767fa2e2107e2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:44.561950 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-31d5bd6f-6679-449b-8c8b-f7af5fd9f2e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23d683cdc90041b5b29767fa2e2107e2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:44.576580 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-31d5bd6f-6679-449b-8c8b-f7af5fd9f2e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23d683cdc90041b5b29767fa2e2107e2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:44.585005 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1290/2570] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:05:44 2022] DELETE /identity/v3/users/23d683cdc90041b5b29767fa2e2107e2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:44.846173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b8591-1abb-4953-81b4-ebb34b32c6af None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:44.863256 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b8591-1abb-4953-81b4-ebb34b32c6af None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:44.901644 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:44.963042 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48e41d-0fcc-4a24-ae3f-fa8c32a02def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:44.963702 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48e41d-0fcc-4a24-ae3f-fa8c32a02def admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:44.964104 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48e41d-0fcc-4a24-ae3f-fa8c32a02def admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:44.964226 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48e41d-0fcc-4a24-ae3f-fa8c32a02def admin admin] PATH_INFO: `/v3/projects/aef8a9d089c9446697837d5ebe2154c8` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:44.965941 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48e41d-0fcc-4a24-ae3f-fa8c32a02def admin admin] RBAC: Authorizing `identity:delete_project(project_id=aef8a9d089c9446697837d5ebe2154c8)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:44.967960 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d48e41d-0fcc-4a24-ae3f-fa8c32a02def admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:45.044592 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6d48e41d-0fcc-4a24-ae3f-fa8c32a02def admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aef8a9d089c9446697837d5ebe2154c8', 'request_id': 'req-6d48e41d-0fcc-4a24-ae3f-fa8c32a02def'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:45.053632 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6d48e41d-0fcc-4a24-ae3f-fa8c32a02def admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aef8a9d089c9446697837d5ebe2154c8', 'request_id': 'req-6d48e41d-0fcc-4a24-ae3f-fa8c32a02def'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:45.060901 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6d48e41d-0fcc-4a24-ae3f-fa8c32a02def admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aef8a9d089c9446697837d5ebe2154c8', 'request_id': 'req-6d48e41d-0fcc-4a24-ae3f-fa8c32a02def'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:45.073814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6d48e41d-0fcc-4a24-ae3f-fa8c32a02def admin admin] The token cache is being invalidate because project aef8a9d089c9446697837d5ebe2154c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:05:45.074148 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6d48e41d-0fcc-4a24-ae3f-fa8c32a02def admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:45.075097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6d48e41d-0fcc-4a24-ae3f-fa8c32a02def admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:45.078547 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1281/2571] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:05:44 2022] DELETE /identity/v3/projects/aef8a9d089c9446697837d5ebe2154c8 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:45.084457 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19c6c6-4f94-4a2a-a186-7cbb9dd1e93c None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:45.085207 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19c6c6-4f94-4a2a-a186-7cbb9dd1e93c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:45.085829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19c6c6-4f94-4a2a-a186-7cbb9dd1e93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:45.114127 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-3f19c6c6-4f94-4a2a-a186-7cbb9dd1e93c None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:45.115409 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3f19c6c6-4f94-4a2a-a186-7cbb9dd1e93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:45.172041 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:45.182279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1291/2572] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:45.190430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad4d0-0131-4cb3-b1a5-9b3cf59a28be None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:45.190710 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad4d0-0131-4cb3-b1a5-9b3cf59a28be None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:45.191021 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad4d0-0131-4cb3-b1a5-9b3cf59a28be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:45.213236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-0e3ad4d0-0131-4cb3-b1a5-9b3cf59a28be None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:45.214491 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-0e3ad4d0-0131-4cb3-b1a5-9b3cf59a28be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:45.223767 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1282/2573] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:45.228200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19c6c6-4f94-4a2a-a186-7cbb9dd1e93c None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:45.238907 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19c6c6-4f94-4a2a-a186-7cbb9dd1e93c None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:45.245296 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c8085-42dc-4ac6-8e2d-918b3db44238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:45.246123 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c8085-42dc-4ac6-8e2d-918b3db44238 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:45.246315 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c8085-42dc-4ac6-8e2d-918b3db44238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:45.247138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c8085-42dc-4ac6-8e2d-918b3db44238 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:45.247293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c8085-42dc-4ac6-8e2d-918b3db44238 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:05:45.247542 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c8085-42dc-4ac6-8e2d-918b3db44238 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:45.249264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c8085-42dc-4ac6-8e2d-918b3db44238 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:45.264320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1292/2574] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 15:05:45.269367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ad4d0-0131-4cb3-b1a5-9b3cf59a28be None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:45.286654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ad4d0-0131-4cb3-b1a5-9b3cf59a28be None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:45.291557 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6377cc-fe02-44ee-971d-7079aee580b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:45.292203 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6377cc-fe02-44ee-971d-7079aee580b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:45.292819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6377cc-fe02-44ee-971d-7079aee580b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:45.293355 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6377cc-fe02-44ee-971d-7079aee580b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:45.294198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6377cc-fe02-44ee-971d-7079aee580b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:45.296688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6377cc-fe02-44ee-971d-7079aee580b8 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:45.335851 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c8085-42dc-4ac6-8e2d-918b3db44238 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:45.337045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1283/2575] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:05:45 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:45.337679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-5b1c8085-42dc-4ac6-8e2d-918b3db44238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:45.341654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6377cc-fe02-44ee-971d-7079aee580b8 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:45.403070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c8085-42dc-4ac6-8e2d-918b3db44238 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:45.409184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6377cc-fe02-44ee-971d-7079aee580b8 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:45.414052 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b43dfd-86f7-46d5-939c-3932afeee7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:45.414664 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b43dfd-86f7-46d5-939c-3932afeee7b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:45.414970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b43dfd-86f7-46d5-939c-3932afeee7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:45.415241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b43dfd-86f7-46d5-939c-3932afeee7b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:45.416314 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b43dfd-86f7-46d5-939c-3932afeee7b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:45.418256 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b43dfd-86f7-46d5-939c-3932afeee7b3 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:45.444029 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1284/2576] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:05:45 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:05:45.445479 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b43dfd-86f7-46d5-939c-3932afeee7b3 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:45.457538 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0747f82-bbe4-4623-92ae-5c0a4d224256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:45.458268 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0747f82-bbe4-4623-92ae-5c0a4d224256 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:45.458791 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0747f82-bbe4-4623-92ae-5c0a4d224256 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:45.459525 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0747f82-bbe4-4623-92ae-5c0a4d224256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:45.461724 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b43dfd-86f7-46d5-939c-3932afeee7b3 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:45.462084 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d0747f82-bbe4-4623-92ae-5c0a4d224256 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:45.467861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebba35-283d-401b-aeba-502e0afef1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:45.468842 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebba35-283d-401b-aeba-502e0afef1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:45.469121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebba35-283d-401b-aeba-502e0afef1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:45.469408 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebba35-283d-401b-aeba-502e0afef1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:45.470243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebba35-283d-401b-aeba-502e0afef1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:45.471941 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebba35-283d-401b-aeba-502e0afef1d8 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:45.476758 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0747f82-bbe4-4623-92ae-5c0a4d224256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:45.478271 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0747f82-bbe4-4623-92ae-5c0a4d224256 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:45.483275 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1285/2577] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:45.487179 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebba35-283d-401b-aeba-502e0afef1d8 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:45.491690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1293/2578] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:05:45.492388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:45.497804 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebba35-283d-401b-aeba-502e0afef1d8 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:45.503906 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55926dd9-2af2-4203-96b1-05f6626c258c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:45.504798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55926dd9-2af2-4203-96b1-05f6626c258c admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:45.505217 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55926dd9-2af2-4203-96b1-05f6626c258c admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:45.505801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55926dd9-2af2-4203-96b1-05f6626c258c admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:45.506339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55926dd9-2af2-4203-96b1-05f6626c258c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:45.508067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55926dd9-2af2-4203-96b1-05f6626c258c admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:45.519302 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1286/2579] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:05:45.523698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0747f82-bbe4-4623-92ae-5c0a4d224256 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:45.533859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0747f82-bbe4-4623-92ae-5c0a4d224256 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:45.538538 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dac12-066c-4ebd-ad34-c2732aa25c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:45.539227 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dac12-066c-4ebd-ad34-c2732aa25c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:45.539503 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dac12-066c-4ebd-ad34-c2732aa25c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:45.539771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dac12-066c-4ebd-ad34-c2732aa25c84 admin admin] PATH_INFO: `/v3/projects/152196822dda4fb698e9e1a738f0e525/users/019e9419b0d941d9b4a33c72b4c486a6/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:45.555124 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dac12-066c-4ebd-ad34-c2732aa25c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=152196822dda4fb698e9e1a738f0e525, user_id=019e9419b0d941d9b4a33c72b4c486a6, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:45.557677 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dac12-066c-4ebd-ad34-c2732aa25c84 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:45.573068 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1294/2580] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:45 2022] PUT /identity/v3/projects/152196822dda4fb698e9e1a738f0e525/users/019e9419b0d941d9b4a33c72b4c486a6/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:45.584760 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f063af-f9f1-4022-bacf-6cde2c47e943 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:45.585046 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f063af-f9f1-4022-bacf-6cde2c47e943 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:45.585320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f063af-f9f1-4022-bacf-6cde2c47e943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:45.641831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-37f063af-f9f1-4022-bacf-6cde2c47e943 None None] MFA Rules not processed for user `019e9419b0d941d9b4a33c72b4c486a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:45.643460 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-37f063af-f9f1-4022-bacf-6cde2c47e943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:45.737620 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1287/2581] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:45.749014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dac12-066c-4ebd-ad34-c2732aa25c84 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:45.750867 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-047dac12-066c-4ebd-ad34-c2732aa25c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:45.859413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dac12-066c-4ebd-ad34-c2732aa25c84 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:45.868925 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5560d38-ae8a-4a45-af7f-5fd9c754153a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:45.870136 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5560d38-ae8a-4a45-af7f-5fd9c754153a service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:45.870283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5560d38-ae8a-4a45-af7f-5fd9c754153a service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:45.870485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5560d38-ae8a-4a45-af7f-5fd9c754153a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:45.881083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5560d38-ae8a-4a45-af7f-5fd9c754153a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:45.882889 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5560d38-ae8a-4a45-af7f-5fd9c754153a service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:45.892986 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1295/2582] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:48.853876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f063af-f9f1-4022-bacf-6cde2c47e943 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:48.971609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f063af-f9f1-4022-bacf-6cde2c47e943 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:48.978885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48072667-afa3-4af9-bc88-9a4233c221c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:48.979827 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48072667-afa3-4af9-bc88-9a4233c221c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:48.980814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48072667-afa3-4af9-bc88-9a4233c221c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:48.981280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48072667-afa3-4af9-bc88-9a4233c221c4 admin admin] PATH_INFO: `/v3/users/019e9419b0d941d9b4a33c72b4c486a6` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:48.983356 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48072667-afa3-4af9-bc88-9a4233c221c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=019e9419b0d941d9b4a33c72b4c486a6)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:48.985694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48072667-afa3-4af9-bc88-9a4233c221c4 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:49.071294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-48072667-afa3-4af9-bc88-9a4233c221c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '019e9419b0d941d9b4a33c72b4c486a6'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.082547 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-48072667-afa3-4af9-bc88-9a4233c221c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '019e9419b0d941d9b4a33c72b4c486a6'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.084662 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-48072667-afa3-4af9-bc88-9a4233c221c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '019e9419b0d941d9b4a33c72b4c486a6'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.102951 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-48072667-afa3-4af9-bc88-9a4233c221c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '019e9419b0d941d9b4a33c72b4c486a6'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.103772 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-48072667-afa3-4af9-bc88-9a4233c221c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '019e9419b0d941d9b4a33c72b4c486a6'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.116709 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:49.120730 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1288/2583] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:05:48 2022] DELETE /identity/v3/users/019e9419b0d941d9b4a33c72b4c486a6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:05:49.125708 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e0157-fce4-4b82-8764-18a136630ca4 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:49.126430 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e0157-fce4-4b82-8764-18a136630ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:49.126430 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e0157-fce4-4b82-8764-18a136630ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:49.143961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:49.153945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-018e0157-fce4-4b82-8764-18a136630ca4 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:49.155703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-018e0157-fce4-4b82-8764-18a136630ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:49.262969 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1296/2584] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:49.270979 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48072667-afa3-4af9-bc88-9a4233c221c4 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:49.272883 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-48072667-afa3-4af9-bc88-9a4233c221c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:49.390166 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48072667-afa3-4af9-bc88-9a4233c221c4 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:49.394847 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787afe1c-520d-41e5-92ac-48b0daa25cb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:49.395579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787afe1c-520d-41e5-92ac-48b0daa25cb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:49.395887 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787afe1c-520d-41e5-92ac-48b0daa25cb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:49.396191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787afe1c-520d-41e5-92ac-48b0daa25cb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:49.410794 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787afe1c-520d-41e5-92ac-48b0daa25cb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:49.411234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787afe1c-520d-41e5-92ac-48b0daa25cb6 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:49.417055 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e0157-fce4-4b82-8764-18a136630ca4 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:49.419048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-018e0157-fce4-4b82-8764-18a136630ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:49.425607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1289/2585] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:49.432441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e0157-fce4-4b82-8764-18a136630ca4 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:49.437523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2caee3-7b90-4036-9320-9770bf1330d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:49.438166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2caee3-7b90-4036-9320-9770bf1330d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:49.438435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2caee3-7b90-4036-9320-9770bf1330d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:49.438699 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2caee3-7b90-4036-9320-9770bf1330d4 admin admin] PATH_INFO: `/v3/users/da004a191bcd43b6a0865919813ec1a8` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:49.439867 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2caee3-7b90-4036-9320-9770bf1330d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da004a191bcd43b6a0865919813ec1a8)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:49.441790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2caee3-7b90-4036-9320-9770bf1330d4 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:49.506030 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-2a2caee3-7b90-4036-9320-9770bf1330d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da004a191bcd43b6a0865919813ec1a8'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.515880 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-2a2caee3-7b90-4036-9320-9770bf1330d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da004a191bcd43b6a0865919813ec1a8'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.524244 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-2a2caee3-7b90-4036-9320-9770bf1330d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da004a191bcd43b6a0865919813ec1a8'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.538074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-2a2caee3-7b90-4036-9320-9770bf1330d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da004a191bcd43b6a0865919813ec1a8'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.538872 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-2a2caee3-7b90-4036-9320-9770bf1330d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da004a191bcd43b6a0865919813ec1a8'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.546311 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1297/2586] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:05:49 2022] DELETE /identity/v3/users/da004a191bcd43b6a0865919813ec1a8 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:05:49.761178 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787afe1c-520d-41e5-92ac-48b0daa25cb6 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:49.765525 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-787afe1c-520d-41e5-92ac-48b0daa25cb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:49.767213 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2caee3-7b90-4036-9320-9770bf1330d4 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:49.768703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2a2caee3-7b90-4036-9320-9770bf1330d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:49.784838 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787afe1c-520d-41e5-92ac-48b0daa25cb6 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:49.862114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2caee3-7b90-4036-9320-9770bf1330d4 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:49.863194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ec720-1217-461b-9fdd-72ac39fd9e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:49.863858 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ec720-1217-461b-9fdd-72ac39fd9e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:49.864148 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ec720-1217-461b-9fdd-72ac39fd9e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:49.864397 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ec720-1217-461b-9fdd-72ac39fd9e23 admin admin] PATH_INFO: `/v3/projects/152196822dda4fb698e9e1a738f0e525` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:49.865565 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ec720-1217-461b-9fdd-72ac39fd9e23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=152196822dda4fb698e9e1a738f0e525)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:49.867363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6dfd3-8591-4796-a619-ac0088893476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:49.867521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ec720-1217-461b-9fdd-72ac39fd9e23 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:49.868156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6dfd3-8591-4796-a619-ac0088893476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:49.868506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6dfd3-8591-4796-a619-ac0088893476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:49.868841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6dfd3-8591-4796-a619-ac0088893476 admin admin] PATH_INFO: `/v3/projects/1c7d5abbf8274516bf79312e3fa22f6c` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:49.870150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6dfd3-8591-4796-a619-ac0088893476 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7d5abbf8274516bf79312e3fa22f6c)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:49.872340 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6dfd3-8591-4796-a619-ac0088893476 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:49.898792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:49.922846 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ae8ec720-1217-461b-9fdd-72ac39fd9e23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '152196822dda4fb698e9e1a738f0e525', 'request_id': 'req-ae8ec720-1217-461b-9fdd-72ac39fd9e23'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.926443 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-54c6dfd3-8591-4796-a619-ac0088893476 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7d5abbf8274516bf79312e3fa22f6c', 'request_id': 'req-54c6dfd3-8591-4796-a619-ac0088893476'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.931609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ae8ec720-1217-461b-9fdd-72ac39fd9e23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '152196822dda4fb698e9e1a738f0e525', 'request_id': 'req-ae8ec720-1217-461b-9fdd-72ac39fd9e23'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.933730 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-54c6dfd3-8591-4796-a619-ac0088893476 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7d5abbf8274516bf79312e3fa22f6c', 'request_id': 'req-54c6dfd3-8591-4796-a619-ac0088893476'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.939870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ae8ec720-1217-461b-9fdd-72ac39fd9e23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '152196822dda4fb698e9e1a738f0e525', 'request_id': 'req-ae8ec720-1217-461b-9fdd-72ac39fd9e23'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.940805 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-54c6dfd3-8591-4796-a619-ac0088893476 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7d5abbf8274516bf79312e3fa22f6c', 'request_id': 'req-54c6dfd3-8591-4796-a619-ac0088893476'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.958052 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-54c6dfd3-8591-4796-a619-ac0088893476 admin admin] The token cache is being invalidate because project 1c7d5abbf8274516bf79312e3fa22f6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:05:49.959355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-54c6dfd3-8591-4796-a619-ac0088893476 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.960040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-54c6dfd3-8591-4796-a619-ac0088893476 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.961661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1298/2587] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:05:49 2022] DELETE /identity/v3/projects/1c7d5abbf8274516bf79312e3fa22f6c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:49.962490 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ae8ec720-1217-461b-9fdd-72ac39fd9e23 admin admin] The token cache is being invalidate because project 152196822dda4fb698e9e1a738f0e525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:05:49.962836 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ae8ec720-1217-461b-9fdd-72ac39fd9e23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.963531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-ae8ec720-1217-461b-9fdd-72ac39fd9e23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:49.967092 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1290/2588] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:05:49 2022] DELETE /identity/v3/projects/152196822dda4fb698e9e1a738f0e525 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:49.967678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb676f9-80e7-4ebe-87d4-941d84f273ad None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:49.968054 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb676f9-80e7-4ebe-87d4-941d84f273ad None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:49.968468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb676f9-80e7-4ebe-87d4-941d84f273ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:49.971609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779fccb-c7ff-448d-8a7e-f05cd1d007ee None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:49.971890 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779fccb-c7ff-448d-8a7e-f05cd1d007ee None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:49.972173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6779fccb-c7ff-448d-8a7e-f05cd1d007ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:49.991368 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-9bb676f9-80e7-4ebe-87d4-941d84f273ad None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:49.992587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-9bb676f9-80e7-4ebe-87d4-941d84f273ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:49.993862 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-6779fccb-c7ff-448d-8a7e-f05cd1d007ee None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:49.995220 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-6779fccb-c7ff-448d-8a7e-f05cd1d007ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:50.036621 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:50.062765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1299/2589] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:05:50.066408 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107430c-1f0a-4a30-bc81-7d614c331e7a None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.066693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107430c-1f0a-4a30-bc81-7d614c331e7a None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.066986 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107430c-1f0a-4a30-bc81-7d614c331e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.076763 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1291/2590] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:05:50.078404 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086ce38-ce0d-4fec-bb9e-a47ebd82f0b4 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.078774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086ce38-ce0d-4fec-bb9e-a47ebd82f0b4 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.079542 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086ce38-ce0d-4fec-bb9e-a47ebd82f0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.091263 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-0107430c-1f0a-4a30-bc81-7d614c331e7a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:50.092524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0107430c-1f0a-4a30-bc81-7d614c331e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:50.101243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1300/2591] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:05:50.105469 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107430c-1f0a-4a30-bc81-7d614c331e7a None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.111446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-f086ce38-ce0d-4fec-bb9e-a47ebd82f0b4 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:50.112666 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-f086ce38-ce0d-4fec-bb9e-a47ebd82f0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:50.114530 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107430c-1f0a-4a30-bc81-7d614c331e7a None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.119072 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bba2f-d121-4e61-b1a2-9cfa727ea27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.119665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bba2f-d121-4e61-b1a2-9cfa727ea27c admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.119934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bba2f-d121-4e61-b1a2-9cfa727ea27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.120195 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07bba2f-d121-4e61-b1a2-9cfa727ea27c admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.121243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1292/2592] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:50.121471 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bba2f-d121-4e61-b1a2-9cfa727ea27c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:05:50.121643 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bba2f-d121-4e61-b1a2-9cfa727ea27c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.123238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07bba2f-d121-4e61-b1a2-9cfa727ea27c admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.125330 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086ce38-ce0d-4fec-bb9e-a47ebd82f0b4 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.134454 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086ce38-ce0d-4fec-bb9e-a47ebd82f0b4 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.135187 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1301/2593] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 15:05:50.139948 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bba2f-d121-4e61-b1a2-9cfa727ea27c admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.140290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73390a5f-e7aa-486b-a3c2-a34059675dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.140887 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73390a5f-e7aa-486b-a3c2-a34059675dbd admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.141167 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73390a5f-e7aa-486b-a3c2-a34059675dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.141438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73390a5f-e7aa-486b-a3c2-a34059675dbd admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.142094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73390a5f-e7aa-486b-a3c2-a34059675dbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:05:50.142386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73390a5f-e7aa-486b-a3c2-a34059675dbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.144006 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73390a5f-e7aa-486b-a3c2-a34059675dbd admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.149494 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07bba2f-d121-4e61-b1a2-9cfa727ea27c admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.154160 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16816d20-bb94-434e-a450-5b441dfa76b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.154768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16816d20-bb94-434e-a450-5b441dfa76b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.155046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16816d20-bb94-434e-a450-5b441dfa76b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.155315 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16816d20-bb94-434e-a450-5b441dfa76b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.155815 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1293/2594] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 15:05:50.156058 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16816d20-bb94-434e-a450-5b441dfa76b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.157784 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16816d20-bb94-434e-a450-5b441dfa76b0 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.160061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73390a5f-e7aa-486b-a3c2-a34059675dbd admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.170056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73390a5f-e7aa-486b-a3c2-a34059675dbd admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.175734 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518215d-9092-4475-bfab-3c0530c69c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.176523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518215d-9092-4475-bfab-3c0530c69c14 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.176942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518215d-9092-4475-bfab-3c0530c69c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.177221 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518215d-9092-4475-bfab-3c0530c69c14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.177974 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518215d-9092-4475-bfab-3c0530c69c14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.180063 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518215d-9092-4475-bfab-3c0530c69c14 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.188462 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1302/2595] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:05:50 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:50.189478 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16816d20-bb94-434e-a450-5b441dfa76b0 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.212887 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1294/2596] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:05:50 2022] POST /identity/v3/projects => generated 361 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:05:50.213899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518215d-9092-4475-bfab-3c0530c69c14 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.267262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16816d20-bb94-434e-a450-5b441dfa76b0 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.278802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:50.285449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518215d-9092-4475-bfab-3c0530c69c14 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.290490 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91559081-6ad1-4d1c-886b-da95cbee5365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.291790 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91559081-6ad1-4d1c-886b-da95cbee5365 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.291790 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91559081-6ad1-4d1c-886b-da95cbee5365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.291790 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91559081-6ad1-4d1c-886b-da95cbee5365 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.292284 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91559081-6ad1-4d1c-886b-da95cbee5365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.294219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91559081-6ad1-4d1c-886b-da95cbee5365 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.316692 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1295/2597] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:05:50 2022] POST /identity/v3/users => generated 281 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:50.320451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91559081-6ad1-4d1c-886b-da95cbee5365 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.331937 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91559081-6ad1-4d1c-886b-da95cbee5365 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.337235 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d008b-3ced-4fa8-a9bd-1fb71349d92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.337913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d008b-3ced-4fa8-a9bd-1fb71349d92e admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.338141 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d008b-3ced-4fa8-a9bd-1fb71349d92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.338409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d008b-3ced-4fa8-a9bd-1fb71349d92e admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.339207 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d008b-3ced-4fa8-a9bd-1fb71349d92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.339607 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138724d-a0c0-4cf0-be49-e27ca230ed96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.340510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138724d-a0c0-4cf0-be49-e27ca230ed96 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.340864 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d008b-3ced-4fa8-a9bd-1fb71349d92e admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.341013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138724d-a0c0-4cf0-be49-e27ca230ed96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.341332 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138724d-a0c0-4cf0-be49-e27ca230ed96 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.341943 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138724d-a0c0-4cf0-be49-e27ca230ed96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.343882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138724d-a0c0-4cf0-be49-e27ca230ed96 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.353395 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1296/2598] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:05:50.357320 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d008b-3ced-4fa8-a9bd-1fb71349d92e admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.366833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1303/2599] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:05:50 2022] POST /identity/v3/users => generated 283 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:50.372141 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138724d-a0c0-4cf0-be49-e27ca230ed96 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.372417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d008b-3ced-4fa8-a9bd-1fb71349d92e admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.377127 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9316bd9-650f-4aac-994d-883033a10f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.377880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9316bd9-650f-4aac-994d-883033a10f71 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.378044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9316bd9-650f-4aac-994d-883033a10f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.378317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9316bd9-650f-4aac-994d-883033a10f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.378986 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9316bd9-650f-4aac-994d-883033a10f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.380570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9316bd9-650f-4aac-994d-883033a10f71 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.383741 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138724d-a0c0-4cf0-be49-e27ca230ed96 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.395399 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8108995-666f-434f-97f1-a0d0bc0d6db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.401553 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1297/2600] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:05:50.403392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9316bd9-650f-4aac-994d-883033a10f71 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.408690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8108995-666f-434f-97f1-a0d0bc0d6db2 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.410631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8108995-666f-434f-97f1-a0d0bc0d6db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.411070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8108995-666f-434f-97f1-a0d0bc0d6db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.412192 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8108995-666f-434f-97f1-a0d0bc0d6db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.415823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8108995-666f-434f-97f1-a0d0bc0d6db2 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.419939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9316bd9-650f-4aac-994d-883033a10f71 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.425812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010c41e-8b5c-44ef-a2a3-80879883225a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.426553 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010c41e-8b5c-44ef-a2a3-80879883225a admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.426787 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010c41e-8b5c-44ef-a2a3-80879883225a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.427059 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010c41e-8b5c-44ef-a2a3-80879883225a admin admin] PATH_INFO: `/v3/projects/ff083303712b4ea7b20a9f02892b9b3e/users/b943143b4574416e8238d7044567a900/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.438631 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1304/2601] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:50.442791 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8108995-666f-434f-97f1-a0d0bc0d6db2 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.443754 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010c41e-8b5c-44ef-a2a3-80879883225a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff083303712b4ea7b20a9f02892b9b3e, user_id=b943143b4574416e8238d7044567a900, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.446426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010c41e-8b5c-44ef-a2a3-80879883225a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.453963 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8108995-666f-434f-97f1-a0d0bc0d6db2 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.461413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1298/2602] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:50 2022] PUT /identity/v3/projects/ff083303712b4ea7b20a9f02892b9b3e/users/b943143b4574416e8238d7044567a900/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:50.471823 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff050e9-d85a-4476-bb27-4fc74817d376 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.472219 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff050e9-d85a-4476-bb27-4fc74817d376 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.472498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff050e9-d85a-4476-bb27-4fc74817d376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.501887 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-dff050e9-d85a-4476-bb27-4fc74817d376 None None] MFA Rules not processed for user `b943143b4574416e8238d7044567a900`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:50.503187 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-dff050e9-d85a-4476-bb27-4fc74817d376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:50.515545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9321108-8c9b-4ec0-bd8f-5181297daf52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.516239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9321108-8c9b-4ec0-bd8f-5181297daf52 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.516526 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9321108-8c9b-4ec0-bd8f-5181297daf52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.516792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9321108-8c9b-4ec0-bd8f-5181297daf52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.517432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9321108-8c9b-4ec0-bd8f-5181297daf52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.519066 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9321108-8c9b-4ec0-bd8f-5181297daf52 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.527627 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:50.529270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1305/2603] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:05:50.533348 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9321108-8c9b-4ec0-bd8f-5181297daf52 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.544033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9321108-8c9b-4ec0-bd8f-5181297daf52 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.549331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46893da-9b89-413a-b304-93cec8d169ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.549965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46893da-9b89-413a-b304-93cec8d169ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.550246 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46893da-9b89-413a-b304-93cec8d169ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.550523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46893da-9b89-413a-b304-93cec8d169ed admin admin] PATH_INFO: `/v3/projects/d6dbb5b652c346b5a518b7fb49270834/users/5cd93ddfaf9b479bb2210a20cf6c090b/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.566311 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46893da-9b89-413a-b304-93cec8d169ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6dbb5b652c346b5a518b7fb49270834, user_id=5cd93ddfaf9b479bb2210a20cf6c090b, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.572530 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46893da-9b89-413a-b304-93cec8d169ed admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.574877 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1299/2604] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:05:50.580933 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff050e9-d85a-4476-bb27-4fc74817d376 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.591745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1306/2605] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:50 2022] PUT /identity/v3/projects/d6dbb5b652c346b5a518b7fb49270834/users/5cd93ddfaf9b479bb2210a20cf6c090b/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:50.596725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff050e9-d85a-4476-bb27-4fc74817d376 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.602007 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81988582-e221-44ef-b6f2-1677ec65e88a None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.602373 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81988582-e221-44ef-b6f2-1677ec65e88a None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.602716 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81988582-e221-44ef-b6f2-1677ec65e88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.631965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-81988582-e221-44ef-b6f2-1677ec65e88a None None] MFA Rules not processed for user `5cd93ddfaf9b479bb2210a20cf6c090b`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:50.633191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-81988582-e221-44ef-b6f2-1677ec65e88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:50.657469 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e4f3f-1fc6-4187-9384-92237b0d3265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.658092 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e4f3f-1fc6-4187-9384-92237b0d3265 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.658386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e4f3f-1fc6-4187-9384-92237b0d3265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.658654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e4f3f-1fc6-4187-9384-92237b0d3265 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.659361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e4f3f-1fc6-4187-9384-92237b0d3265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.661261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e4f3f-1fc6-4187-9384-92237b0d3265 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.693363 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1300/2606] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:05:50 2022] POST /identity/v3/projects => generated 363 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:05:50.693574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:50.694629 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e4f3f-1fc6-4187-9384-92237b0d3265 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.704293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1307/2607] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:05:50.708487 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81988582-e221-44ef-b6f2-1677ec65e88a None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.719352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81988582-e221-44ef-b6f2-1677ec65e88a None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.775537 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e4f3f-1fc6-4187-9384-92237b0d3265 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.781187 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1d9ee-91e6-473d-ae49-1fd4f49b3009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.782173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1d9ee-91e6-473d-ae49-1fd4f49b3009 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.782632 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1d9ee-91e6-473d-ae49-1fd4f49b3009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.783650 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f1d9ee-91e6-473d-ae49-1fd4f49b3009 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.783971 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f1d9ee-91e6-473d-ae49-1fd4f49b3009 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.786049 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f1d9ee-91e6-473d-ae49-1fd4f49b3009 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.796974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc978e-dd4b-4b81-bba5-00464296ff43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.797885 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc978e-dd4b-4b81-bba5-00464296ff43 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.798302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc978e-dd4b-4b81-bba5-00464296ff43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.798759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc978e-dd4b-4b81-bba5-00464296ff43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.799763 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc978e-dd4b-4b81-bba5-00464296ff43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.802953 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc978e-dd4b-4b81-bba5-00464296ff43 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.814155 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1301/2608] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:05:50 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:50.818537 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1d9ee-91e6-473d-ae49-1fd4f49b3009 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.829967 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f1d9ee-91e6-473d-ae49-1fd4f49b3009 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.836800 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76061fea-d499-47d4-8753-f6d8d9b818cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.837406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76061fea-d499-47d4-8753-f6d8d9b818cf admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.837684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76061fea-d499-47d4-8753-f6d8d9b818cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.837953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76061fea-d499-47d4-8753-f6d8d9b818cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.838596 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76061fea-d499-47d4-8753-f6d8d9b818cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.840851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76061fea-d499-47d4-8753-f6d8d9b818cf admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.846842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1308/2609] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:05:50 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:05:50.847975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc978e-dd4b-4b81-bba5-00464296ff43 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.852969 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1302/2610] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:05:50.857722 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76061fea-d499-47d4-8753-f6d8d9b818cf admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.868147 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76061fea-d499-47d4-8753-f6d8d9b818cf admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.894578 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:50.913987 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc978e-dd4b-4b81-bba5-00464296ff43 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.920000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03869b8e-c1bf-4910-9c19-d7f49b751832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.920667 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03869b8e-c1bf-4910-9c19-d7f49b751832 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.920963 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03869b8e-c1bf-4910-9c19-d7f49b751832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.921269 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03869b8e-c1bf-4910-9c19-d7f49b751832 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.921972 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03869b8e-c1bf-4910-9c19-d7f49b751832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.924402 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03869b8e-c1bf-4910-9c19-d7f49b751832 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.938897 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e840020-fe88-477e-a364-d6abb6f02ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.939607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e840020-fe88-477e-a364-d6abb6f02ed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.939885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e840020-fe88-477e-a364-d6abb6f02ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.940153 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e840020-fe88-477e-a364-d6abb6f02ed3 admin admin] PATH_INFO: `/v3/projects/93c53e75e8d6436a9b237863a614cc75/users/33495362318f4be68f602fdd4f50f9a4/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.948291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1309/2611] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:05:50 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:05:50.952276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03869b8e-c1bf-4910-9c19-d7f49b751832 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.957859 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e840020-fe88-477e-a364-d6abb6f02ed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c53e75e8d6436a9b237863a614cc75, user_id=33495362318f4be68f602fdd4f50f9a4, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.960833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e840020-fe88-477e-a364-d6abb6f02ed3 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.962810 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03869b8e-c1bf-4910-9c19-d7f49b751832 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:50.969525 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174483c-e178-48dc-a65a-4aca7ef1aa66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:50.970129 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174483c-e178-48dc-a65a-4aca7ef1aa66 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:50.970483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174483c-e178-48dc-a65a-4aca7ef1aa66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:50.970672 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174483c-e178-48dc-a65a-4aca7ef1aa66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:50.971591 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174483c-e178-48dc-a65a-4aca7ef1aa66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:50.973468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174483c-e178-48dc-a65a-4aca7ef1aa66 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:50.976063 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1303/2612] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:50 2022] PUT /identity/v3/projects/93c53e75e8d6436a9b237863a614cc75/users/33495362318f4be68f602fdd4f50f9a4/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:50.980536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e840020-fe88-477e-a364-d6abb6f02ed3 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.985435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1310/2613] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:05:50.989590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174483c-e178-48dc-a65a-4aca7ef1aa66 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:50.999898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174483c-e178-48dc-a65a-4aca7ef1aa66 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:51.048928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e840020-fe88-477e-a364-d6abb6f02ed3 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:51.054904 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6aa47-1797-434e-b1ae-289cd041c1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:51.055507 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6aa47-1797-434e-b1ae-289cd041c1c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:51.055786 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6aa47-1797-434e-b1ae-289cd041c1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:51.056063 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6aa47-1797-434e-b1ae-289cd041c1c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:51.056708 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6aa47-1797-434e-b1ae-289cd041c1c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:51.058312 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6aa47-1797-434e-b1ae-289cd041c1c7 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:51.065546 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a8b34-7677-46c1-80d4-bb5f55404bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:51.066241 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a8b34-7677-46c1-80d4-bb5f55404bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:51.066481 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a8b34-7677-46c1-80d4-bb5f55404bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:51.067004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a8b34-7677-46c1-80d4-bb5f55404bd8 admin admin] PATH_INFO: `/v3/projects/72cd9e1f07a640fa863d086eb693f577/users/f95385278a8a44a39211e1a79b006635/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:51.069976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:51.072880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1304/2614] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:05:51.081043 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6aa47-1797-434e-b1ae-289cd041c1c7 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:51.098642 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6aa47-1797-434e-b1ae-289cd041c1c7 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:51.106837 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df80a7-41d4-406b-8a5f-b91926135662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:51.107490 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a8b34-7677-46c1-80d4-bb5f55404bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72cd9e1f07a640fa863d086eb693f577, user_id=f95385278a8a44a39211e1a79b006635, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:51.107922 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df80a7-41d4-406b-8a5f-b91926135662 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:51.108190 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df80a7-41d4-406b-8a5f-b91926135662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:51.108563 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df80a7-41d4-406b-8a5f-b91926135662 admin admin] PATH_INFO: `/v3/projects/93c53e75e8d6436a9b237863a614cc75/users/33495362318f4be68f602fdd4f50f9a4/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:51.110430 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a8b34-7677-46c1-80d4-bb5f55404bd8 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:51.110831 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df80a7-41d4-406b-8a5f-b91926135662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c53e75e8d6436a9b237863a614cc75, user_id=33495362318f4be68f602fdd4f50f9a4, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:51.113992 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df80a7-41d4-406b-8a5f-b91926135662 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:51.126244 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1311/2615] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:50 2022] PUT /identity/v3/projects/72cd9e1f07a640fa863d086eb693f577/users/f95385278a8a44a39211e1a79b006635/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:05:51.127164 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1305/2616] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:51 2022] PUT /identity/v3/projects/93c53e75e8d6436a9b237863a614cc75/users/33495362318f4be68f602fdd4f50f9a4/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:51.130245 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a8b34-7677-46c1-80d4-bb5f55404bd8 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:51.139131 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f903fe-e3d4-42b9-9030-de004896c7e8 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:51.139405 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f903fe-e3d4-42b9-9030-de004896c7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:51.139689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f903fe-e3d4-42b9-9030-de004896c7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:51.170142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-c8f903fe-e3d4-42b9-9030-de004896c7e8 None None] MFA Rules not processed for user `33495362318f4be68f602fdd4f50f9a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:51.171410 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-c8f903fe-e3d4-42b9-9030-de004896c7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:51.203284 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a8b34-7677-46c1-80d4-bb5f55404bd8 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:51.260875 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1306/2617] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:05:51.274597 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f903fe-e3d4-42b9-9030-de004896c7e8 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:51.276222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-c8f903fe-e3d4-42b9-9030-de004896c7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:51.276653 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507fdbf9-f2ca-4615-98a5-7a970c43de0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:51.278037 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fdbf9-f2ca-4615-98a5-7a970c43de0b admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:51.278174 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fdbf9-f2ca-4615-98a5-7a970c43de0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:51.278358 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507fdbf9-f2ca-4615-98a5-7a970c43de0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:51.279043 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507fdbf9-f2ca-4615-98a5-7a970c43de0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:51.280680 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507fdbf9-f2ca-4615-98a5-7a970c43de0b admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:51.294788 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1312/2618] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:05:51.296487 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507fdbf9-f2ca-4615-98a5-7a970c43de0b admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:51.306513 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:51.307396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507fdbf9-f2ca-4615-98a5-7a970c43de0b admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:51.313046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894583f5-78f5-471b-8be2-f83a6038cdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:51.313738 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894583f5-78f5-471b-8be2-f83a6038cdb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:51.314017 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894583f5-78f5-471b-8be2-f83a6038cdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:51.314292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894583f5-78f5-471b-8be2-f83a6038cdb5 admin admin] PATH_INFO: `/v3/projects/72cd9e1f07a640fa863d086eb693f577/users/f95385278a8a44a39211e1a79b006635/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:51.316278 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894583f5-78f5-471b-8be2-f83a6038cdb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72cd9e1f07a640fa863d086eb693f577, user_id=f95385278a8a44a39211e1a79b006635, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:51.319130 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894583f5-78f5-471b-8be2-f83a6038cdb5 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:51.333102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1313/2619] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:51 2022] PUT /identity/v3/projects/72cd9e1f07a640fa863d086eb693f577/users/f95385278a8a44a39211e1a79b006635/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:51.345911 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4cf0f-4a95-47d0-9905-ef88e7a3d5f6 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:51.346172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4cf0f-4a95-47d0-9905-ef88e7a3d5f6 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:51.346470 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4cf0f-4a95-47d0-9905-ef88e7a3d5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:51.354905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f903fe-e3d4-42b9-9030-de004896c7e8 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:51.376381 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-7ae4cf0f-4a95-47d0-9905-ef88e7a3d5f6 None None] MFA Rules not processed for user `f95385278a8a44a39211e1a79b006635`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:51.377622 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7ae4cf0f-4a95-47d0-9905-ef88e7a3d5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:51.418429 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeced73-b741-460b-a973-cc62bd34a1af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:51.419295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeced73-b741-460b-a973-cc62bd34a1af service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:51.419714 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeced73-b741-460b-a973-cc62bd34a1af service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:51.419999 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeced73-b741-460b-a973-cc62bd34a1af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:51.471552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:51.473614 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1314/2620] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:05:51.492492 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeced73-b741-460b-a973-cc62bd34a1af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:51.493927 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeced73-b741-460b-a973-cc62bd34a1af service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:51.504522 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1307/2621] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:05:51.531042 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4cf0f-4a95-47d0-9905-ef88e7a3d5f6 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:51.538776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7ae4cf0f-4a95-47d0-9905-ef88e7a3d5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:51.549704 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4cf0f-4a95-47d0-9905-ef88e7a3d5f6 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:51.553894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f9668-7f68-4b58-8ce4-e18c86293c91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:51.554523 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f9668-7f68-4b58-8ce4-e18c86293c91 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:51.554838 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f9668-7f68-4b58-8ce4-e18c86293c91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:51.555106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f9668-7f68-4b58-8ce4-e18c86293c91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:51.565541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f9668-7f68-4b58-8ce4-e18c86293c91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:51.566982 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f9668-7f68-4b58-8ce4-e18c86293c91 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:51.577727 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1315/2622] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:51 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:51.680118 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeced73-b741-460b-a973-cc62bd34a1af service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:51.705583 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeced73-b741-460b-a973-cc62bd34a1af service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:51.711846 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fdd0b-2b41-46ca-9d06-4644726c95eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:51.712631 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fdd0b-2b41-46ca-9d06-4644726c95eb service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:51.712968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fdd0b-2b41-46ca-9d06-4644726c95eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:51.713303 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fdd0b-2b41-46ca-9d06-4644726c95eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:51.793020 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0fdd0b-2b41-46ca-9d06-4644726c95eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:51.795100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0fdd0b-2b41-46ca-9d06-4644726c95eb service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:51.806390 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1308/2623] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:51.877001 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f9668-7f68-4b58-8ce4-e18c86293c91 service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:51.878574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-0c8f9668-7f68-4b58-8ce4-e18c86293c91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:51.964089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f9668-7f68-4b58-8ce4-e18c86293c91 service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:51.971450 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a486e6-dcd3-4bc1-9c3d-72f1766b1e8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:51.971967 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a486e6-dcd3-4bc1-9c3d-72f1766b1e8a service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:51.972242 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a486e6-dcd3-4bc1-9c3d-72f1766b1e8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:51.972511 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a486e6-dcd3-4bc1-9c3d-72f1766b1e8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:51.974502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a3a486e6-dcd3-4bc1-9c3d-72f1766b1e8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:52.040200 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fdd0b-2b41-46ca-9d06-4644726c95eb service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:52.065556 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a486e6-dcd3-4bc1-9c3d-72f1766b1e8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:52.067071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a486e6-dcd3-4bc1-9c3d-72f1766b1e8a service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:52.070632 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:52.077131 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1316/2624] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:52.115277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fdd0b-2b41-46ca-9d06-4644726c95eb service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:52.120084 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47b2c2-6b07-40cf-ae9f-0ffded29f761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:52.121449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47b2c2-6b07-40cf-ae9f-0ffded29f761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:52.121930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47b2c2-6b07-40cf-ae9f-0ffded29f761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:52.123112 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47b2c2-6b07-40cf-ae9f-0ffded29f761 admin admin] PATH_INFO: `/v3/users/b943143b4574416e8238d7044567a900` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:52.124324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47b2c2-6b07-40cf-ae9f-0ffded29f761 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b943143b4574416e8238d7044567a900)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:52.126275 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47b2c2-6b07-40cf-ae9f-0ffded29f761 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:52.184749 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5f47b2c2-6b07-40cf-ae9f-0ffded29f761 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b943143b4574416e8238d7044567a900'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:52.193935 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5f47b2c2-6b07-40cf-ae9f-0ffded29f761 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b943143b4574416e8238d7044567a900'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:52.194854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5f47b2c2-6b07-40cf-ae9f-0ffded29f761 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b943143b4574416e8238d7044567a900'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:52.213015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5f47b2c2-6b07-40cf-ae9f-0ffded29f761 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b943143b4574416e8238d7044567a900'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:52.213754 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5f47b2c2-6b07-40cf-ae9f-0ffded29f761 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b943143b4574416e8238d7044567a900'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:52.227889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1309/2625] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:05:52 2022] DELETE /identity/v3/users/b943143b4574416e8238d7044567a900 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:52.232718 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e6af8-d89c-431e-bd39-c7e6d6b76e01 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:52.233243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e6af8-d89c-431e-bd39-c7e6d6b76e01 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:52.233243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e6af8-d89c-431e-bd39-c7e6d6b76e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:52.253531 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-2f5e6af8-d89c-431e-bd39-c7e6d6b76e01 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:52.256025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2f5e6af8-d89c-431e-bd39-c7e6d6b76e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:52.336444 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1317/2626] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:05:52.347745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47b2c2-6b07-40cf-ae9f-0ffded29f761 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:52.349258 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-5f47b2c2-6b07-40cf-ae9f-0ffded29f761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:52.423067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47b2c2-6b07-40cf-ae9f-0ffded29f761 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:52.426753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ca75f-acd1-487e-8166-0a3cae3491ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:52.427358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ca75f-acd1-487e-8166-0a3cae3491ed service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:52.427638 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ca75f-acd1-487e-8166-0a3cae3491ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:52.427909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ca75f-acd1-487e-8166-0a3cae3491ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:52.437681 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ca75f-acd1-487e-8166-0a3cae3491ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:52.439245 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ca75f-acd1-487e-8166-0a3cae3491ed service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:52.449451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1310/2627] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:52.903298 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e6af8-d89c-431e-bd39-c7e6d6b76e01 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:52.905173 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2f5e6af8-d89c-431e-bd39-c7e6d6b76e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:52.918243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e6af8-d89c-431e-bd39-c7e6d6b76e01 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:52.923545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807c8e7-292a-4625-9824-eeccaaf3dd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:52.924220 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807c8e7-292a-4625-9824-eeccaaf3dd1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:52.924461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807c8e7-292a-4625-9824-eeccaaf3dd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:52.924735 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807c8e7-292a-4625-9824-eeccaaf3dd1c admin admin] PATH_INFO: `/v3/projects/ff083303712b4ea7b20a9f02892b9b3e` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:52.925853 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807c8e7-292a-4625-9824-eeccaaf3dd1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff083303712b4ea7b20a9f02892b9b3e)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:52.927782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807c8e7-292a-4625-9824-eeccaaf3dd1c admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:52.970237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:52.985882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-e807c8e7-292a-4625-9824-eeccaaf3dd1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff083303712b4ea7b20a9f02892b9b3e', 'request_id': 'req-e807c8e7-292a-4625-9824-eeccaaf3dd1c'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:52.992753 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-e807c8e7-292a-4625-9824-eeccaaf3dd1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff083303712b4ea7b20a9f02892b9b3e', 'request_id': 'req-e807c8e7-292a-4625-9824-eeccaaf3dd1c'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:53.007236 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-e807c8e7-292a-4625-9824-eeccaaf3dd1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff083303712b4ea7b20a9f02892b9b3e', 'request_id': 'req-e807c8e7-292a-4625-9824-eeccaaf3dd1c'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:53.031276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-e807c8e7-292a-4625-9824-eeccaaf3dd1c admin admin] The token cache is being invalidate because project ff083303712b4ea7b20a9f02892b9b3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:05:53.031560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-e807c8e7-292a-4625-9824-eeccaaf3dd1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:53.032497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-e807c8e7-292a-4625-9824-eeccaaf3dd1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:53.034424 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1318/2628] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:05:52 2022] DELETE /identity/v3/projects/ff083303712b4ea7b20a9f02892b9b3e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:53.038015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ca75f-acd1-487e-8166-0a3cae3491ed service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:53.049171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ca75f-acd1-487e-8166-0a3cae3491ed service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:53.104177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:53.123925 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990b9f9-ed2e-4856-87cb-975c04bfe620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:53.124857 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990b9f9-ed2e-4856-87cb-975c04bfe620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:53.125134 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990b9f9-ed2e-4856-87cb-975c04bfe620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:53.125408 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990b9f9-ed2e-4856-87cb-975c04bfe620 admin admin] PATH_INFO: `/v3/users/33495362318f4be68f602fdd4f50f9a4` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:53.126640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990b9f9-ed2e-4856-87cb-975c04bfe620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33495362318f4be68f602fdd4f50f9a4)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:53.128779 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990b9f9-ed2e-4856-87cb-975c04bfe620 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:53.204833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d990b9f9-ed2e-4856-87cb-975c04bfe620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33495362318f4be68f602fdd4f50f9a4'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:53.216881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d990b9f9-ed2e-4856-87cb-975c04bfe620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33495362318f4be68f602fdd4f50f9a4'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:53.220505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d990b9f9-ed2e-4856-87cb-975c04bfe620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33495362318f4be68f602fdd4f50f9a4'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:53.237889 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d990b9f9-ed2e-4856-87cb-975c04bfe620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33495362318f4be68f602fdd4f50f9a4'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:53.238590 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-d990b9f9-ed2e-4856-87cb-975c04bfe620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33495362318f4be68f602fdd4f50f9a4'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:53.257388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1311/2629] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:05:53 2022] DELETE /identity/v3/users/33495362318f4be68f602fdd4f50f9a4 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:53.885482 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807c8e7-292a-4625-9824-eeccaaf3dd1c admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:53.886959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e807c8e7-292a-4625-9824-eeccaaf3dd1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:53.901053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807c8e7-292a-4625-9824-eeccaaf3dd1c admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:53.970182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f0788-fe8a-49a7-81d3-804ce9b397a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:53.970951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f0788-fe8a-49a7-81d3-804ce9b397a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:53.971234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f0788-fe8a-49a7-81d3-804ce9b397a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:53.971506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f0788-fe8a-49a7-81d3-804ce9b397a4 admin admin] PATH_INFO: `/v3/projects/93c53e75e8d6436a9b237863a614cc75` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:53.972638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f0788-fe8a-49a7-81d3-804ce9b397a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93c53e75e8d6436a9b237863a614cc75)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:53.974637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f0788-fe8a-49a7-81d3-804ce9b397a4 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:54.035863 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-432f0788-fe8a-49a7-81d3-804ce9b397a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93c53e75e8d6436a9b237863a614cc75', 'request_id': 'req-432f0788-fe8a-49a7-81d3-804ce9b397a4'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:54.042132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-432f0788-fe8a-49a7-81d3-804ce9b397a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93c53e75e8d6436a9b237863a614cc75', 'request_id': 'req-432f0788-fe8a-49a7-81d3-804ce9b397a4'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:54.049191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-432f0788-fe8a-49a7-81d3-804ce9b397a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93c53e75e8d6436a9b237863a614cc75', 'request_id': 'req-432f0788-fe8a-49a7-81d3-804ce9b397a4'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:54.062774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-432f0788-fe8a-49a7-81d3-804ce9b397a4 admin admin] The token cache is being invalidate because project 93c53e75e8d6436a9b237863a614cc75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:05:54.063095 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-432f0788-fe8a-49a7-81d3-804ce9b397a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:54.063759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-432f0788-fe8a-49a7-81d3-804ce9b397a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:05:54.065602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1319/2630] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:05:53 2022] DELETE /identity/v3/projects/93c53e75e8d6436a9b237863a614cc75 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:54.071708 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eb0af-6767-4a08-a23d-2cd53216fdfd None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:54.072121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eb0af-6767-4a08-a23d-2cd53216fdfd None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:54.072273 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eb0af-6767-4a08-a23d-2cd53216fdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:54.097557 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-a08eb0af-6767-4a08-a23d-2cd53216fdfd None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:54.098944 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a08eb0af-6767-4a08-a23d-2cd53216fdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:54.165461 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1312/2631] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:54.172489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490372e3-e7e7-497c-be37-215ba63b1ad0 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:54.172775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490372e3-e7e7-497c-be37-215ba63b1ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:54.173046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490372e3-e7e7-497c-be37-215ba63b1ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:54.194300 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-490372e3-e7e7-497c-be37-215ba63b1ad0 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:54.195629 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-490372e3-e7e7-497c-be37-215ba63b1ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:54.206294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1320/2632] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:05:54.211036 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08eb0af-6767-4a08-a23d-2cd53216fdfd None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:54.224213 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08eb0af-6767-4a08-a23d-2cd53216fdfd None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:54.230767 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb3bad-8a81-4ba3-881c-1e7076203450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:54.231561 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb3bad-8a81-4ba3-881c-1e7076203450 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:54.232045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb3bad-8a81-4ba3-881c-1e7076203450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:54.232370 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb3bad-8a81-4ba3-881c-1e7076203450 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:54.233069 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb3bad-8a81-4ba3-881c-1e7076203450 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:05:54.233335 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb3bad-8a81-4ba3-881c-1e7076203450 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:54.253652 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb3bad-8a81-4ba3-881c-1e7076203450 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:54.277293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490372e3-e7e7-497c-be37-215ba63b1ad0 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:54.283025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1313/2633] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:05:54.290796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490372e3-e7e7-497c-be37-215ba63b1ad0 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:54.297261 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe14146-e27f-4459-a886-b6e72140a66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:54.298049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe14146-e27f-4459-a886-b6e72140a66b admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:54.298771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe14146-e27f-4459-a886-b6e72140a66b admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:54.299135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe14146-e27f-4459-a886-b6e72140a66b admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:54.299880 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe14146-e27f-4459-a886-b6e72140a66b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:54.302398 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe14146-e27f-4459-a886-b6e72140a66b admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:54.349830 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1321/2634] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:05:54 2022] POST /identity/v3/projects => generated 355 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:05:54.353886 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb3bad-8a81-4ba3-881c-1e7076203450 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:54.364967 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb3bad-8a81-4ba3-881c-1e7076203450 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:54.426006 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:54.439711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f69386-0063-4fd9-ad1e-94c351120c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:54.441548 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f69386-0063-4fd9-ad1e-94c351120c45 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:54.442093 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f69386-0063-4fd9-ad1e-94c351120c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:54.442555 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f69386-0063-4fd9-ad1e-94c351120c45 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:54.443645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f69386-0063-4fd9-ad1e-94c351120c45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:54.445951 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f69386-0063-4fd9-ad1e-94c351120c45 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:54.482306 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1314/2635] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:05:54 2022] POST /identity/v3/users => generated 278 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:05:54.483849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe14146-e27f-4459-a886-b6e72140a66b admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:54.502242 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe14146-e27f-4459-a886-b6e72140a66b admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:54.539791 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e497fba-9750-42f0-b9f4-4179bec0e31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:54.540396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e497fba-9750-42f0-b9f4-4179bec0e31f admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:54.540634 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e497fba-9750-42f0-b9f4-4179bec0e31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:54.540892 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e497fba-9750-42f0-b9f4-4179bec0e31f admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:54.541608 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e497fba-9750-42f0-b9f4-4179bec0e31f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:54.543225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e497fba-9750-42f0-b9f4-4179bec0e31f admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:54.574191 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1322/2636] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:54.577902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f69386-0063-4fd9-ad1e-94c351120c45 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:54.594147 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f69386-0063-4fd9-ad1e-94c351120c45 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:54.611248 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161ef69-fe2d-4591-b45d-dd3861724524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:54.611890 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161ef69-fe2d-4591-b45d-dd3861724524 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:54.612165 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161ef69-fe2d-4591-b45d-dd3861724524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:54.612617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161ef69-fe2d-4591-b45d-dd3861724524 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:54.613094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161ef69-fe2d-4591-b45d-dd3861724524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:54.614893 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161ef69-fe2d-4591-b45d-dd3861724524 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:54.639462 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1315/2637] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:54.643941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e497fba-9750-42f0-b9f4-4179bec0e31f admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:54.670991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e497fba-9750-42f0-b9f4-4179bec0e31f admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:54.689427 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e9691-9a4e-440a-8768-bbb191cc5233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:54.691791 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e9691-9a4e-440a-8768-bbb191cc5233 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:54.692944 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e9691-9a4e-440a-8768-bbb191cc5233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:54.693742 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e9691-9a4e-440a-8768-bbb191cc5233 admin admin] PATH_INFO: `/v3/projects/fe05a44283164990b3f411e8e2f19b4c/users/65d658e7417c422c8700a0435f0caba2/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:54.703957 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:54.737627 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4e9691-9a4e-440a-8768-bbb191cc5233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe05a44283164990b3f411e8e2f19b4c, user_id=65d658e7417c422c8700a0435f0caba2, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:54.741533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4e9691-9a4e-440a-8768-bbb191cc5233 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:54.761053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1323/2638] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:54 2022] PUT /identity/v3/projects/fe05a44283164990b3f411e8e2f19b4c/users/65d658e7417c422c8700a0435f0caba2/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:54.779192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0e724-4674-4fc5-b0ce-21d4e7290ef2 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:54.779664 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0e724-4674-4fc5-b0ce-21d4e7290ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:54.780093 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0e724-4674-4fc5-b0ce-21d4e7290ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:54.834354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-c2b0e724-4674-4fc5-b0ce-21d4e7290ef2 None None] MFA Rules not processed for user `65d658e7417c422c8700a0435f0caba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:54.835825 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-c2b0e724-4674-4fc5-b0ce-21d4e7290ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:54.959772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e9691-9a4e-440a-8768-bbb191cc5233 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:54.968214 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1316/2639] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:55.103050 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e9691-9a4e-440a-8768-bbb191cc5233 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:55.110833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6135fa8-6bed-48f0-a3a2-1a0948170e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:55.111964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6135fa8-6bed-48f0-a3a2-1a0948170e8a admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:55.111964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6135fa8-6bed-48f0-a3a2-1a0948170e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:55.111964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6135fa8-6bed-48f0-a3a2-1a0948170e8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:55.113270 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6135fa8-6bed-48f0-a3a2-1a0948170e8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:55.115977 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6135fa8-6bed-48f0-a3a2-1a0948170e8a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:55.171021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1324/2640] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:05:54 2022] POST /identity/v3/projects => generated 357 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:05:55.173443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0e724-4674-4fc5-b0ce-21d4e7290ef2 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:55.190587 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0e724-4674-4fc5-b0ce-21d4e7290ef2 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:55.319559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83b90d-2399-49c9-8fc0-61a75d8b5610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:55.320156 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83b90d-2399-49c9-8fc0-61a75d8b5610 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:55.320548 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83b90d-2399-49c9-8fc0-61a75d8b5610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:55.320916 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83b90d-2399-49c9-8fc0-61a75d8b5610 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:55.321660 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83b90d-2399-49c9-8fc0-61a75d8b5610 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:55.324519 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83b90d-2399-49c9-8fc0-61a75d8b5610 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:55.362530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1317/2641] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:05:55 2022] POST /identity/v3/users => generated 285 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:05:55.366856 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6135fa8-6bed-48f0-a3a2-1a0948170e8a admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:55.387117 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6135fa8-6bed-48f0-a3a2-1a0948170e8a admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:55.403441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708525b3-5daf-41e2-b810-e6ff63a97053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:55.404158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708525b3-5daf-41e2-b810-e6ff63a97053 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:55.404504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708525b3-5daf-41e2-b810-e6ff63a97053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:55.404843 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708525b3-5daf-41e2-b810-e6ff63a97053 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:55.405558 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708525b3-5daf-41e2-b810-e6ff63a97053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:55.407298 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708525b3-5daf-41e2-b810-e6ff63a97053 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:55.427844 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1325/2642] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:55.433832 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83b90d-2399-49c9-8fc0-61a75d8b5610 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:55.446451 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83b90d-2399-49c9-8fc0-61a75d8b5610 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:55.458105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c9f55-1608-4a95-a40a-3144e0b37d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:55.463061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c9f55-1608-4a95-a40a-3144e0b37d6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:55.463431 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c9f55-1608-4a95-a40a-3144e0b37d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:55.463776 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c9f55-1608-4a95-a40a-3144e0b37d6f admin admin] PATH_INFO: `/v3/projects/ab1087e94a8e4736b1a8e612ec5616fe/users/69bcec515d484037ab8afdc6defa067a/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:55.484907 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c9f55-1608-4a95-a40a-3144e0b37d6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab1087e94a8e4736b1a8e612ec5616fe, user_id=69bcec515d484037ab8afdc6defa067a, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:55.488536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c9f55-1608-4a95-a40a-3144e0b37d6f admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:55.514047 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1318/2643] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:55 2022] PUT /identity/v3/projects/ab1087e94a8e4736b1a8e612ec5616fe/users/69bcec515d484037ab8afdc6defa067a/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:55.518628 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708525b3-5daf-41e2-b810-e6ff63a97053 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:55.535125 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708525b3-5daf-41e2-b810-e6ff63a97053 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:55.616597 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba761b7b-46ca-4a6e-b8d2-24ca5ba3bab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:55.617344 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba761b7b-46ca-4a6e-b8d2-24ca5ba3bab1 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:55.617769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba761b7b-46ca-4a6e-b8d2-24ca5ba3bab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:55.618123 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba761b7b-46ca-4a6e-b8d2-24ca5ba3bab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:55.618857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba761b7b-46ca-4a6e-b8d2-24ca5ba3bab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:55.621135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba761b7b-46ca-4a6e-b8d2-24ca5ba3bab1 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:55.645002 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1326/2644] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:05:55.646128 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c9f55-1608-4a95-a40a-3144e0b37d6f admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:55.657674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c9f55-1608-4a95-a40a-3144e0b37d6f admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:55.662954 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9e92f-d773-434e-a095-743ceda98ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:55.663631 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9e92f-d773-434e-a095-743ceda98ff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:55.663898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9e92f-d773-434e-a095-743ceda98ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:55.664204 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9e92f-d773-434e-a095-743ceda98ff0 admin admin] PATH_INFO: `/v3/projects/ab1087e94a8e4736b1a8e612ec5616fe/users/69bcec515d484037ab8afdc6defa067a/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:55.666701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9e92f-d773-434e-a095-743ceda98ff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab1087e94a8e4736b1a8e612ec5616fe, user_id=69bcec515d484037ab8afdc6defa067a, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:55.669948 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9e92f-d773-434e-a095-743ceda98ff0 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:55.684058 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1319/2645] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:05:55 2022] PUT /identity/v3/projects/ab1087e94a8e4736b1a8e612ec5616fe/users/69bcec515d484037ab8afdc6defa067a/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:05:55.695415 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c89a2-f3e9-4d93-88c1-2a43247db385 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:55.695698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c89a2-f3e9-4d93-88c1-2a43247db385 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:55.695974 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c89a2-f3e9-4d93-88c1-2a43247db385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:55.725598 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-8d8c89a2-f3e9-4d93-88c1-2a43247db385 None None] MFA Rules not processed for user `69bcec515d484037ab8afdc6defa067a`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:05:55.726846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-8d8c89a2-f3e9-4d93-88c1-2a43247db385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:55.781696 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:55.812297 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1327/2646] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:05:55 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:05:55.827048 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9e92f-d773-434e-a095-743ceda98ff0 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:55.829056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-0ee9e92f-d773-434e-a095-743ceda98ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:05:55.847137 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:05:55.895545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9e92f-d773-434e-a095-743ceda98ff0 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:55.900879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4e04b-da8d-4d96-afb4-a8c14072d60b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:55.901698 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4e04b-da8d-4d96-afb4-a8c14072d60b service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:55.902012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4e04b-da8d-4d96-afb4-a8c14072d60b service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:55.902375 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4e04b-da8d-4d96-afb4-a8c14072d60b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:55.912514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4e04b-da8d-4d96-afb4-a8c14072d60b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:55.913950 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4e04b-da8d-4d96-afb4-a8c14072d60b service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:55.924508 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1320/2647] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:55 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:05:56.004473 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c89a2-f3e9-4d93-88c1-2a43247db385 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:05:56.014573 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c89a2-f3e9-4d93-88c1-2a43247db385 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:05:56.018913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c750f2-0547-4e41-ac5b-f96d390ad514 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:05:56.019691 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c750f2-0547-4e41-ac5b-f96d390ad514 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:05:56.019979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c750f2-0547-4e41-ac5b-f96d390ad514 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:05:56.020237 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c750f2-0547-4e41-ac5b-f96d390ad514 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:05:56.086051 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c750f2-0547-4e41-ac5b-f96d390ad514 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:05:56.087615 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c750f2-0547-4e41-ac5b-f96d390ad514 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:05:56.097976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1328/2648] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:05:56 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:02.894652 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4e04b-da8d-4d96-afb4-a8c14072d60b service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:02.896231 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-03e4e04b-da8d-4d96-afb4-a8c14072d60b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:02.963918 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4e04b-da8d-4d96-afb4-a8c14072d60b service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:02.968510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f99f4-8a77-4c4a-bc33-f80d981df30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:02.969143 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f99f4-8a77-4c4a-bc33-f80d981df30c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:02.969417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f99f4-8a77-4c4a-bc33-f80d981df30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:02.969687 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f99f4-8a77-4c4a-bc33-f80d981df30c admin admin] PATH_INFO: `/v3/users/5cd93ddfaf9b479bb2210a20cf6c090b` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:02.970834 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f99f4-8a77-4c4a-bc33-f80d981df30c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cd93ddfaf9b479bb2210a20cf6c090b)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:02.972822 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f99f4-8a77-4c4a-bc33-f80d981df30c admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:03.027161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3e4f99f4-8a77-4c4a-bc33-f80d981df30c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cd93ddfaf9b479bb2210a20cf6c090b'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:03.035293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3e4f99f4-8a77-4c4a-bc33-f80d981df30c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cd93ddfaf9b479bb2210a20cf6c090b'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:03.036020 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3e4f99f4-8a77-4c4a-bc33-f80d981df30c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cd93ddfaf9b479bb2210a20cf6c090b'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:03.049817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3e4f99f4-8a77-4c4a-bc33-f80d981df30c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cd93ddfaf9b479bb2210a20cf6c090b'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:03.050658 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3e4f99f4-8a77-4c4a-bc33-f80d981df30c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cd93ddfaf9b479bb2210a20cf6c090b'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:03.064525 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1321/2649] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:02 2022] DELETE /identity/v3/users/5cd93ddfaf9b479bb2210a20cf6c090b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:03.068681 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458fc59-18b8-485a-88dd-cbe3535ded18 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:03.069217 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458fc59-18b8-485a-88dd-cbe3535ded18 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:03.072434 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458fc59-18b8-485a-88dd-cbe3535ded18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:03.097019 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-d458fc59-18b8-485a-88dd-cbe3535ded18 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:03.098243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d458fc59-18b8-485a-88dd-cbe3535ded18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:03.162771 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1329/2650] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:06:03.172484 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f99f4-8a77-4c4a-bc33-f80d981df30c admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:03.174061 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3e4f99f4-8a77-4c4a-bc33-f80d981df30c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:03.224981 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:03.250228 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f99f4-8a77-4c4a-bc33-f80d981df30c admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:03.254288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac2e34-809b-4f5c-b97e-a282f209b7e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:03.255020 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac2e34-809b-4f5c-b97e-a282f209b7e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:03.255298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac2e34-809b-4f5c-b97e-a282f209b7e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:03.255564 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac2e34-809b-4f5c-b97e-a282f209b7e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:03.266798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac2e34-809b-4f5c-b97e-a282f209b7e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:03.268447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac2e34-809b-4f5c-b97e-a282f209b7e6 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:03.280863 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1322/2651] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:03.764154 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458fc59-18b8-485a-88dd-cbe3535ded18 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:03.766312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d458fc59-18b8-485a-88dd-cbe3535ded18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:03.780956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458fc59-18b8-485a-88dd-cbe3535ded18 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:03.786854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:03.787507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:03.787780 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:03.788061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a admin admin] PATH_INFO: `/v3/projects/d6dbb5b652c346b5a518b7fb49270834` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:03.789521 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6dbb5b652c346b5a518b7fb49270834)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:03.792661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:03.872486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6dbb5b652c346b5a518b7fb49270834', 'request_id': 'req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:03.881987 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6dbb5b652c346b5a518b7fb49270834', 'request_id': 'req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:03.891548 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6dbb5b652c346b5a518b7fb49270834', 'request_id': 'req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:03.913586 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a admin admin] The token cache is being invalidate because project d6dbb5b652c346b5a518b7fb49270834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:03.915064 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:03.916269 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:03.919906 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1330/2652] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:03 2022] DELETE /identity/v3/projects/d6dbb5b652c346b5a518b7fb49270834 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:03.923664 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac2e34-809b-4f5c-b97e-a282f209b7e6 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:03.941774 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac2e34-809b-4f5c-b97e-a282f209b7e6 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:04.039294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ff8ab-7f8d-4c96-8b15-60b5d4164423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:04.039866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ff8ab-7f8d-4c96-8b15-60b5d4164423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:04.040142 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ff8ab-7f8d-4c96-8b15-60b5d4164423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:04.040414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ff8ab-7f8d-4c96-8b15-60b5d4164423 admin admin] PATH_INFO: `/v3/users/f95385278a8a44a39211e1a79b006635` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:04.041552 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ff8ab-7f8d-4c96-8b15-60b5d4164423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f95385278a8a44a39211e1a79b006635)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:04.043844 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ff8ab-7f8d-4c96-8b15-60b5d4164423 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:04.111776 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9f4ff8ab-7f8d-4c96-8b15-60b5d4164423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f95385278a8a44a39211e1a79b006635'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:04.134854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9f4ff8ab-7f8d-4c96-8b15-60b5d4164423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f95385278a8a44a39211e1a79b006635'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:04.136200 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9f4ff8ab-7f8d-4c96-8b15-60b5d4164423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f95385278a8a44a39211e1a79b006635'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:04.152131 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9f4ff8ab-7f8d-4c96-8b15-60b5d4164423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f95385278a8a44a39211e1a79b006635'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:04.152862 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9f4ff8ab-7f8d-4c96-8b15-60b5d4164423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f95385278a8a44a39211e1a79b006635'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:04.166650 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1323/2653] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:03 2022] DELETE /identity/v3/users/f95385278a8a44a39211e1a79b006635 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:04.657374 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:04.658952 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:04.670316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:04.677276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b428cb6-0bdb-4bd1-9253-c8dc3053832a admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:04.754911 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c44bfa-9a94-405b-a914-9f424f41b720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:04.755556 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c44bfa-9a94-405b-a914-9f424f41b720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:04.755864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c44bfa-9a94-405b-a914-9f424f41b720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:04.756166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c44bfa-9a94-405b-a914-9f424f41b720 admin admin] PATH_INFO: `/v3/projects/72cd9e1f07a640fa863d086eb693f577` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:04.757296 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c44bfa-9a94-405b-a914-9f424f41b720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72cd9e1f07a640fa863d086eb693f577)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:04.759401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c44bfa-9a94-405b-a914-9f424f41b720 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:04.807877 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-35c44bfa-9a94-405b-a914-9f424f41b720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72cd9e1f07a640fa863d086eb693f577', 'request_id': 'req-35c44bfa-9a94-405b-a914-9f424f41b720'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:04.814875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-35c44bfa-9a94-405b-a914-9f424f41b720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72cd9e1f07a640fa863d086eb693f577', 'request_id': 'req-35c44bfa-9a94-405b-a914-9f424f41b720'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:04.821224 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-35c44bfa-9a94-405b-a914-9f424f41b720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72cd9e1f07a640fa863d086eb693f577', 'request_id': 'req-35c44bfa-9a94-405b-a914-9f424f41b720'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:04.848464 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-35c44bfa-9a94-405b-a914-9f424f41b720 admin admin] The token cache is being invalidate because project 72cd9e1f07a640fa863d086eb693f577 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:04.848689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-35c44bfa-9a94-405b-a914-9f424f41b720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:04.849567 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-35c44bfa-9a94-405b-a914-9f424f41b720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:04.851150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1331/2654] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:04 2022] DELETE /identity/v3/projects/72cd9e1f07a640fa863d086eb693f577 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:04.856474 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0ed5f-d785-4818-8f34-de7b07b5d9e2 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:04.856761 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0ed5f-d785-4818-8f34-de7b07b5d9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:04.857039 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0ed5f-d785-4818-8f34-de7b07b5d9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:04.930562 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-00e0ed5f-d785-4818-8f34-de7b07b5d9e2 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:04.931817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-00e0ed5f-d785-4818-8f34-de7b07b5d9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:04.945162 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:05.035811 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1324/2655] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:06:05.043823 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18d30b-f5e0-4002-8355-8c7a6691dacd None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:05.044108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18d30b-f5e0-4002-8355-8c7a6691dacd None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:05.044383 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18d30b-f5e0-4002-8355-8c7a6691dacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:05.064552 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-cc18d30b-f5e0-4002-8355-8c7a6691dacd None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:05.065808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-cc18d30b-f5e0-4002-8355-8c7a6691dacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:05.078782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1332/2656] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:05.082171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0ed5f-d785-4818-8f34-de7b07b5d9e2 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:05.095261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0ed5f-d785-4818-8f34-de7b07b5d9e2 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:05.100182 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42e15b-368d-4e8f-bf2d-9deaa72d2c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:05.100731 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42e15b-368d-4e8f-bf2d-9deaa72d2c88 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:05.101150 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42e15b-368d-4e8f-bf2d-9deaa72d2c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:05.101429 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42e15b-368d-4e8f-bf2d-9deaa72d2c88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:05.102191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42e15b-368d-4e8f-bf2d-9deaa72d2c88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:06:05.102392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42e15b-368d-4e8f-bf2d-9deaa72d2c88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:05.104037 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42e15b-368d-4e8f-bf2d-9deaa72d2c88 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:05.116658 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1325/2657] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:06:05.122167 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18d30b-f5e0-4002-8355-8c7a6691dacd None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:05.134330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18d30b-f5e0-4002-8355-8c7a6691dacd None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:05.142179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fb085-bd2b-4729-9599-8b4301330f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:05.142395 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fb085-bd2b-4729-9599-8b4301330f5e admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:05.142801 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fb085-bd2b-4729-9599-8b4301330f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:05.143181 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fb085-bd2b-4729-9599-8b4301330f5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:05.143915 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fb085-bd2b-4729-9599-8b4301330f5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:05.146138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fb085-bd2b-4729-9599-8b4301330f5e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:05.185293 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1333/2658] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:05 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:05.186240 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42e15b-368d-4e8f-bf2d-9deaa72d2c88 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:05.196853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42e15b-368d-4e8f-bf2d-9deaa72d2c88 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:05.257908 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf619b8-c681-4e39-8fd0-fae18f818bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:05.258518 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf619b8-c681-4e39-8fd0-fae18f818bec admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:05.258825 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf619b8-c681-4e39-8fd0-fae18f818bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:05.259096 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf619b8-c681-4e39-8fd0-fae18f818bec admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:05.259725 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf619b8-c681-4e39-8fd0-fae18f818bec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:05.262188 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf619b8-c681-4e39-8fd0-fae18f818bec admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:05.287197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1326/2659] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:05 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:05.291219 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fb085-bd2b-4729-9599-8b4301330f5e admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:05.301531 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fb085-bd2b-4729-9599-8b4301330f5e admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:05.328146 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c40781-0d48-4310-9e14-9140c0349edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:05.329575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c40781-0d48-4310-9e14-9140c0349edd admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:05.330368 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c40781-0d48-4310-9e14-9140c0349edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:05.330647 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c40781-0d48-4310-9e14-9140c0349edd admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:05.331368 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c40781-0d48-4310-9e14-9140c0349edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:05.332983 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c40781-0d48-4310-9e14-9140c0349edd admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:05.344498 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1334/2660] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:05.348625 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf619b8-c681-4e39-8fd0-fae18f818bec admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:05.359107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf619b8-c681-4e39-8fd0-fae18f818bec admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:05.365333 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda4a00-99d4-423d-a8fc-ea06c3c8c325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:05.365946 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda4a00-99d4-423d-a8fc-ea06c3c8c325 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:05.366224 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda4a00-99d4-423d-a8fc-ea06c3c8c325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:05.367017 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda4a00-99d4-423d-a8fc-ea06c3c8c325 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:05.367661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda4a00-99d4-423d-a8fc-ea06c3c8c325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:05.369416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda4a00-99d4-423d-a8fc-ea06c3c8c325 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:05.383311 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1327/2661] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:05.388100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c40781-0d48-4310-9e14-9140c0349edd admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:05.398061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c40781-0d48-4310-9e14-9140c0349edd admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:05.402944 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c4d39-6b31-498b-8cdf-e7e88e21c790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:05.403728 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c4d39-6b31-498b-8cdf-e7e88e21c790 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:05.404009 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c4d39-6b31-498b-8cdf-e7e88e21c790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:05.404280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c4d39-6b31-498b-8cdf-e7e88e21c790 admin admin] PATH_INFO: `/v3/projects/2d4f63653ac9466289c2a069c899215e/users/d9cb5d3092de45d2981e5c9abc440bd1/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:05.419765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c4d39-6b31-498b-8cdf-e7e88e21c790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d4f63653ac9466289c2a069c899215e, user_id=d9cb5d3092de45d2981e5c9abc440bd1, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:05.422403 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c4d39-6b31-498b-8cdf-e7e88e21c790 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:05.438466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1335/2662] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:05 2022] PUT /identity/v3/projects/2d4f63653ac9466289c2a069c899215e/users/d9cb5d3092de45d2981e5c9abc440bd1/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:05.449814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e0940-27e4-49a4-8aa7-ad5dc75e7534 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:05.449814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e0940-27e4-49a4-8aa7-ad5dc75e7534 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:05.450139 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e0940-27e4-49a4-8aa7-ad5dc75e7534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:05.481806 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-229e0940-27e4-49a4-8aa7-ad5dc75e7534 None None] MFA Rules not processed for user `d9cb5d3092de45d2981e5c9abc440bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:05.484350 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-229e0940-27e4-49a4-8aa7-ad5dc75e7534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:05.520622 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:05.571260 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1328/2663] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:05.576085 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c4d39-6b31-498b-8cdf-e7e88e21c790 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:05.614231 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:05.668002 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c4d39-6b31-498b-8cdf-e7e88e21c790 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:05.673842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06330de-e7fc-4d80-9067-64a0194ab3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:05.674589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06330de-e7fc-4d80-9067-64a0194ab3df admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:05.675123 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06330de-e7fc-4d80-9067-64a0194ab3df admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:05.675320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06330de-e7fc-4d80-9067-64a0194ab3df admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:05.676035 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06330de-e7fc-4d80-9067-64a0194ab3df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:05.678114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06330de-e7fc-4d80-9067-64a0194ab3df admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:05.711360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1336/2664] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:05 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:05.715589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e0940-27e4-49a4-8aa7-ad5dc75e7534 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:05.728510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e0940-27e4-49a4-8aa7-ad5dc75e7534 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:05.787341 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320b839-0d76-47ac-8519-e57f33b7ac24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:05.787945 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320b839-0d76-47ac-8519-e57f33b7ac24 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:05.788244 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320b839-0d76-47ac-8519-e57f33b7ac24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:05.788550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320b839-0d76-47ac-8519-e57f33b7ac24 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:05.789144 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320b839-0d76-47ac-8519-e57f33b7ac24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:05.791051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320b839-0d76-47ac-8519-e57f33b7ac24 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:05.814163 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1329/2665] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:05 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:05.819783 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06330de-e7fc-4d80-9067-64a0194ab3df admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:05.830564 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06330de-e7fc-4d80-9067-64a0194ab3df admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:05.835665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e46d30-02c8-4b7f-b1d6-e5d40e1bb2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:05.836264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e46d30-02c8-4b7f-b1d6-e5d40e1bb2fd admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:05.836537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e46d30-02c8-4b7f-b1d6-e5d40e1bb2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:05.836804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e46d30-02c8-4b7f-b1d6-e5d40e1bb2fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:05.837437 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e46d30-02c8-4b7f-b1d6-e5d40e1bb2fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:05.839596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e46d30-02c8-4b7f-b1d6-e5d40e1bb2fd admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:05.850483 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1337/2666] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:06:05.855854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320b839-0d76-47ac-8519-e57f33b7ac24 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:05.866136 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320b839-0d76-47ac-8519-e57f33b7ac24 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:05.870812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f3d69-1077-42dd-a4ec-396ac8f4d913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:05.871462 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f3d69-1077-42dd-a4ec-396ac8f4d913 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:05.871736 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f3d69-1077-42dd-a4ec-396ac8f4d913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:05.872005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f3d69-1077-42dd-a4ec-396ac8f4d913 admin admin] PATH_INFO: `/v3/projects/44028a3174ab4e51b89c2d7239520982/users/329c4fb8cb704ea6bad5c56b5d789f13/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:05.887633 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f3d69-1077-42dd-a4ec-396ac8f4d913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44028a3174ab4e51b89c2d7239520982, user_id=329c4fb8cb704ea6bad5c56b5d789f13, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:05.890305 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f3d69-1077-42dd-a4ec-396ac8f4d913 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:05.904291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1330/2667] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:05 2022] PUT /identity/v3/projects/44028a3174ab4e51b89c2d7239520982/users/329c4fb8cb704ea6bad5c56b5d789f13/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:05.908289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e46d30-02c8-4b7f-b1d6-e5d40e1bb2fd admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:05.919220 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e46d30-02c8-4b7f-b1d6-e5d40e1bb2fd admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:05.983781 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2eaf4-a32f-4ba2-b891-8b705002d699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:05.984390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2eaf4-a32f-4ba2-b891-8b705002d699 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:05.984672 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2eaf4-a32f-4ba2-b891-8b705002d699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:05.984983 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2eaf4-a32f-4ba2-b891-8b705002d699 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:05.985593 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2eaf4-a32f-4ba2-b891-8b705002d699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:05.987221 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2eaf4-a32f-4ba2-b891-8b705002d699 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:05.997151 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1338/2668] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:06.001512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f3d69-1077-42dd-a4ec-396ac8f4d913 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:06.012545 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f3d69-1077-42dd-a4ec-396ac8f4d913 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:06.017498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b7ae5-0d46-440e-a5c0-8b8cafef00b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:06.018151 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b7ae5-0d46-440e-a5c0-8b8cafef00b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:06.018432 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b7ae5-0d46-440e-a5c0-8b8cafef00b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:06.018705 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b7ae5-0d46-440e-a5c0-8b8cafef00b9 admin admin] PATH_INFO: `/v3/projects/44028a3174ab4e51b89c2d7239520982/users/329c4fb8cb704ea6bad5c56b5d789f13/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:06.020669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b7ae5-0d46-440e-a5c0-8b8cafef00b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44028a3174ab4e51b89c2d7239520982, user_id=329c4fb8cb704ea6bad5c56b5d789f13, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:06.023388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b7ae5-0d46-440e-a5c0-8b8cafef00b9 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:06.034478 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1331/2669] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:06 2022] PUT /identity/v3/projects/44028a3174ab4e51b89c2d7239520982/users/329c4fb8cb704ea6bad5c56b5d789f13/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:06.046488 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d1a02-606c-48db-8f0a-6df155cf0ea8 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:06.046811 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d1a02-606c-48db-8f0a-6df155cf0ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:06.047112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d1a02-606c-48db-8f0a-6df155cf0ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:06.075912 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-952d1a02-606c-48db-8f0a-6df155cf0ea8 None None] MFA Rules not processed for user `329c4fb8cb704ea6bad5c56b5d789f13`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:06.077116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-952d1a02-606c-48db-8f0a-6df155cf0ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:06.147436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1339/2670] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:06.161047 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b7ae5-0d46-440e-a5c0-8b8cafef00b9 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:06.162809 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d87b7ae5-0d46-440e-a5c0-8b8cafef00b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:06.237533 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b7ae5-0d46-440e-a5c0-8b8cafef00b9 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:06.241598 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d37a2-4846-4dfb-9c2d-1ab189cc3abb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:06.242117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d37a2-4846-4dfb-9c2d-1ab189cc3abb service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:06.242397 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d37a2-4846-4dfb-9c2d-1ab189cc3abb service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:06.242670 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d37a2-4846-4dfb-9c2d-1ab189cc3abb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:06.252487 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d37a2-4846-4dfb-9c2d-1ab189cc3abb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:06.253909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d37a2-4846-4dfb-9c2d-1ab189cc3abb service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:06.264581 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1332/2671] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:06.602753 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d1a02-606c-48db-8f0a-6df155cf0ea8 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:06.612156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d1a02-606c-48db-8f0a-6df155cf0ea8 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:06.617073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659b995-a888-43e0-8a59-72da1512909c tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c4fb8cb704ea6bad5c56b5d789f13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44028a3174ab4e51b89c2d7239520982', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:06.617701 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659b995-a888-43e0-8a59-72da1512909c tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:06.618015 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659b995-a888-43e0-8a59-72da1512909c tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:06.618274 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659b995-a888-43e0-8a59-72da1512909c tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:06.618986 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659b995-a888-43e0-8a59-72da1512909c tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:06:06.619298 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659b995-a888-43e0-8a59-72da1512909c tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:06.621492 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659b995-a888-43e0-8a59-72da1512909c tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:06.630989 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:06.638065 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1340/2672] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:06:06.639631 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d37a2-4846-4dfb-9c2d-1ab189cc3abb service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:06.649075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d37a2-4846-4dfb-9c2d-1ab189cc3abb service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:06.653642 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17d07e-5f0e-4c04-829d-e2c794bffa96 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c4fb8cb704ea6bad5c56b5d789f13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44028a3174ab4e51b89c2d7239520982', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:06.654231 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17d07e-5f0e-4c04-829d-e2c794bffa96 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:06.654511 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17d07e-5f0e-4c04-829d-e2c794bffa96 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:06.654818 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17d07e-5f0e-4c04-829d-e2c794bffa96 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:06.655444 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17d07e-5f0e-4c04-829d-e2c794bffa96 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:06.657334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17d07e-5f0e-4c04-829d-e2c794bffa96 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:06.666661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:06.686881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1333/2673] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:06 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:06.691130 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659b995-a888-43e0-8a59-72da1512909c tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:06.701605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659b995-a888-43e0-8a59-72da1512909c tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:06.759520 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a2ec1-5d61-4694-9f04-b8fe537557a0 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c4fb8cb704ea6bad5c56b5d789f13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44028a3174ab4e51b89c2d7239520982', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:06.760134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a2ec1-5d61-4694-9f04-b8fe537557a0 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:06.761096 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a2ec1-5d61-4694-9f04-b8fe537557a0 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:06.761394 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a2ec1-5d61-4694-9f04-b8fe537557a0 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:06.762054 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a2ec1-5d61-4694-9f04-b8fe537557a0 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:06:06.762362 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a2ec1-5d61-4694-9f04-b8fe537557a0 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:06.764026 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a2ec1-5d61-4694-9f04-b8fe537557a0 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:06.776685 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1341/2674] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:06:06.816309 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17d07e-5f0e-4c04-829d-e2c794bffa96 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:06.827268 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17d07e-5f0e-4c04-829d-e2c794bffa96 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:06.832027 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c81340-f19e-4796-a998-726a37dc3c30 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '329c4fb8cb704ea6bad5c56b5d789f13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44028a3174ab4e51b89c2d7239520982', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:06.832775 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c81340-f19e-4796-a998-726a37dc3c30 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:06.833067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c81340-f19e-4796-a998-726a37dc3c30 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:06.833342 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c81340-f19e-4796-a998-726a37dc3c30 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] PATH_INFO: `/v3/projects/d8ab5054fd044f71a70f67c857f503ca` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:06.834821 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c81340-f19e-4796-a998-726a37dc3c30 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d8ab5054fd044f71a70f67c857f503ca)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:06.836822 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c81340-f19e-4796-a998-726a37dc3c30 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:06.886348 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-07c81340-f19e-4796-a998-726a37dc3c30 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8ab5054fd044f71a70f67c857f503ca', 'request_id': 'req-07c81340-f19e-4796-a998-726a37dc3c30'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:06.895318 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-07c81340-f19e-4796-a998-726a37dc3c30 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8ab5054fd044f71a70f67c857f503ca', 'request_id': 'req-07c81340-f19e-4796-a998-726a37dc3c30'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:06.901930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-07c81340-f19e-4796-a998-726a37dc3c30 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8ab5054fd044f71a70f67c857f503ca', 'request_id': 'req-07c81340-f19e-4796-a998-726a37dc3c30'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:06.912732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-07c81340-f19e-4796-a998-726a37dc3c30 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] The token cache is being invalidate because project d8ab5054fd044f71a70f67c857f503ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:06.913022 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-07c81340-f19e-4796-a998-726a37dc3c30 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:06.913766 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-07c81340-f19e-4796-a998-726a37dc3c30 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:06.915891 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1334/2675] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:06 2022] DELETE /identity/v3/projects/d8ab5054fd044f71a70f67c857f503ca => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:07.133032 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a2ec1-5d61-4694-9f04-b8fe537557a0 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:07.134806 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a72a2ec1-5d61-4694-9f04-b8fe537557a0 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:07.201294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a2ec1-5d61-4694-9f04-b8fe537557a0 tempest-VolumeQuotaClassesTest-169593531 tempest-VolumeQuotaClassesTest-169593531-project-admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:07.257221 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c58a8b-7e20-464c-9575-783810be0d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:07.257849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c58a8b-7e20-464c-9575-783810be0d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:07.258129 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c58a8b-7e20-464c-9575-783810be0d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:07.258401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c58a8b-7e20-464c-9575-783810be0d3a admin admin] PATH_INFO: `/v3/users/d9cb5d3092de45d2981e5c9abc440bd1` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:07.259565 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c58a8b-7e20-464c-9575-783810be0d3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9cb5d3092de45d2981e5c9abc440bd1)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:07.261520 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c58a8b-7e20-464c-9575-783810be0d3a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:07.304223 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:07.306760 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-87c58a8b-7e20-464c-9575-783810be0d3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9cb5d3092de45d2981e5c9abc440bd1'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:07.316431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-87c58a8b-7e20-464c-9575-783810be0d3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9cb5d3092de45d2981e5c9abc440bd1'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:07.331632 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-87c58a8b-7e20-464c-9575-783810be0d3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9cb5d3092de45d2981e5c9abc440bd1'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:07.350071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-87c58a8b-7e20-464c-9575-783810be0d3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9cb5d3092de45d2981e5c9abc440bd1'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:07.350958 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-87c58a8b-7e20-464c-9575-783810be0d3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9cb5d3092de45d2981e5c9abc440bd1'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:07.357607 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1342/2676] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:07 2022] DELETE /identity/v3/users/d9cb5d3092de45d2981e5c9abc440bd1 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:07.362084 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd7983-0949-49ff-8189-85dce4b1b37c None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:07.362546 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd7983-0949-49ff-8189-85dce4b1b37c None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:07.362661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd7983-0949-49ff-8189-85dce4b1b37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:07.381417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-7cfd7983-0949-49ff-8189-85dce4b1b37c None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:07.382623 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7cfd7983-0949-49ff-8189-85dce4b1b37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:07.459521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1335/2677] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:07.469868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c58a8b-7e20-464c-9575-783810be0d3a admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:07.471326 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-87c58a8b-7e20-464c-9575-783810be0d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:07.530914 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c58a8b-7e20-464c-9575-783810be0d3a admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:07.534539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e91364-e35e-4f9c-b263-ff542c1e17ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:07.535195 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e91364-e35e-4f9c-b263-ff542c1e17ea service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:07.535476 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e91364-e35e-4f9c-b263-ff542c1e17ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:07.535792 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e91364-e35e-4f9c-b263-ff542c1e17ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:07.545585 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e91364-e35e-4f9c-b263-ff542c1e17ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:07.547048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e91364-e35e-4f9c-b263-ff542c1e17ea service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:07.557159 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1343/2678] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:07.819868 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd7983-0949-49ff-8189-85dce4b1b37c None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:07.821364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7cfd7983-0949-49ff-8189-85dce4b1b37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:07.836392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd7983-0949-49ff-8189-85dce4b1b37c None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:07.843110 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98daf25b-6f5c-4d69-b0cf-ed3700299c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:07.843776 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daf25b-6f5c-4d69-b0cf-ed3700299c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:07.844060 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daf25b-6f5c-4d69-b0cf-ed3700299c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:07.844375 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daf25b-6f5c-4d69-b0cf-ed3700299c45 admin admin] PATH_INFO: `/v3/projects/2d4f63653ac9466289c2a069c899215e` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:07.845757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98daf25b-6f5c-4d69-b0cf-ed3700299c45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d4f63653ac9466289c2a069c899215e)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:07.847772 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98daf25b-6f5c-4d69-b0cf-ed3700299c45 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:07.895499 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-98daf25b-6f5c-4d69-b0cf-ed3700299c45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d4f63653ac9466289c2a069c899215e', 'request_id': 'req-98daf25b-6f5c-4d69-b0cf-ed3700299c45'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:07.903794 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-98daf25b-6f5c-4d69-b0cf-ed3700299c45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d4f63653ac9466289c2a069c899215e', 'request_id': 'req-98daf25b-6f5c-4d69-b0cf-ed3700299c45'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:07.910218 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-98daf25b-6f5c-4d69-b0cf-ed3700299c45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d4f63653ac9466289c2a069c899215e', 'request_id': 'req-98daf25b-6f5c-4d69-b0cf-ed3700299c45'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:07.914904 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=83,keystone:INSERT=3,keystone:DELETE=14 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:07.922304 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-98daf25b-6f5c-4d69-b0cf-ed3700299c45 admin admin] The token cache is being invalidate because project 2d4f63653ac9466289c2a069c899215e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:07.922638 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-98daf25b-6f5c-4d69-b0cf-ed3700299c45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:07.923767 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-98daf25b-6f5c-4d69-b0cf-ed3700299c45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:07.925450 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1336/2679] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:07 2022] DELETE /identity/v3/projects/2d4f63653ac9466289c2a069c899215e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:07.929556 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e91364-e35e-4f9c-b263-ff542c1e17ea service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:07.939636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e91364-e35e-4f9c-b263-ff542c1e17ea service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:07.995317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436d85f-dc25-415b-9be5-4d47f8491736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:07.995945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436d85f-dc25-415b-9be5-4d47f8491736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:07.996226 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436d85f-dc25-415b-9be5-4d47f8491736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:07.996501 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436d85f-dc25-415b-9be5-4d47f8491736 admin admin] PATH_INFO: `/v3/users/329c4fb8cb704ea6bad5c56b5d789f13` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:07.997618 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436d85f-dc25-415b-9be5-4d47f8491736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=329c4fb8cb704ea6bad5c56b5d789f13)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:07.999574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436d85f-dc25-415b-9be5-4d47f8491736 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:08.046040 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a436d85f-dc25-415b-9be5-4d47f8491736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '329c4fb8cb704ea6bad5c56b5d789f13'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:08.052787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a436d85f-dc25-415b-9be5-4d47f8491736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '329c4fb8cb704ea6bad5c56b5d789f13'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:08.060272 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a436d85f-dc25-415b-9be5-4d47f8491736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '329c4fb8cb704ea6bad5c56b5d789f13'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:08.062148 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:08.074859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a436d85f-dc25-415b-9be5-4d47f8491736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '329c4fb8cb704ea6bad5c56b5d789f13'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:08.075464 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a436d85f-dc25-415b-9be5-4d47f8491736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '329c4fb8cb704ea6bad5c56b5d789f13'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:08.081737 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1344/2680] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:07 2022] DELETE /identity/v3/users/329c4fb8cb704ea6bad5c56b5d789f13 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:06:08.368685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98daf25b-6f5c-4d69-b0cf-ed3700299c45 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:08.370853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-98daf25b-6f5c-4d69-b0cf-ed3700299c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:08.384187 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98daf25b-6f5c-4d69-b0cf-ed3700299c45 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:08.443205 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15946f41-ed51-4381-8be4-ebc15ea6bd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:08.443838 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15946f41-ed51-4381-8be4-ebc15ea6bd0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:08.444118 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15946f41-ed51-4381-8be4-ebc15ea6bd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:08.444409 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15946f41-ed51-4381-8be4-ebc15ea6bd0c admin admin] PATH_INFO: `/v3/projects/44028a3174ab4e51b89c2d7239520982` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:08.445503 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15946f41-ed51-4381-8be4-ebc15ea6bd0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=44028a3174ab4e51b89c2d7239520982)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:08.447442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15946f41-ed51-4381-8be4-ebc15ea6bd0c admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:08.496419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-15946f41-ed51-4381-8be4-ebc15ea6bd0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44028a3174ab4e51b89c2d7239520982', 'request_id': 'req-15946f41-ed51-4381-8be4-ebc15ea6bd0c'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:08.503921 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-15946f41-ed51-4381-8be4-ebc15ea6bd0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44028a3174ab4e51b89c2d7239520982', 'request_id': 'req-15946f41-ed51-4381-8be4-ebc15ea6bd0c'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:08.510236 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-15946f41-ed51-4381-8be4-ebc15ea6bd0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44028a3174ab4e51b89c2d7239520982', 'request_id': 'req-15946f41-ed51-4381-8be4-ebc15ea6bd0c'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:08.520981 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-15946f41-ed51-4381-8be4-ebc15ea6bd0c admin admin] The token cache is being invalidate because project 44028a3174ab4e51b89c2d7239520982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:08.521267 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-15946f41-ed51-4381-8be4-ebc15ea6bd0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:08.521907 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-15946f41-ed51-4381-8be4-ebc15ea6bd0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:08.523471 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1337/2681] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:08 2022] DELETE /identity/v3/projects/44028a3174ab4e51b89c2d7239520982 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:08.528730 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d2ffa-4e57-48ff-b3e5-9eabb2e6cbcd None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:08.529018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d2ffa-4e57-48ff-b3e5-9eabb2e6cbcd None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:08.529296 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d2ffa-4e57-48ff-b3e5-9eabb2e6cbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:08.549517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-5e1d2ffa-4e57-48ff-b3e5-9eabb2e6cbcd None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:08.550751 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-5e1d2ffa-4e57-48ff-b3e5-9eabb2e6cbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:08.612382 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1345/2682] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:08.620658 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee915a-8635-44c3-b260-ecef08ae4c6e None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:08.620943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee915a-8635-44c3-b260-ecef08ae4c6e None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:08.621216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee915a-8635-44c3-b260-ecef08ae4c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:08.641872 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-58ee915a-8635-44c3-b260-ecef08ae4c6e None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:08.643126 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-58ee915a-8635-44c3-b260-ecef08ae4c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:08.651152 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1338/2683] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:08.655447 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d2ffa-4e57-48ff-b3e5-9eabb2e6cbcd None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:08.665120 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d2ffa-4e57-48ff-b3e5-9eabb2e6cbcd None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:08.670200 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85326e4c-bb5b-4ee2-9494-71a9801a17ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:08.670842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85326e4c-bb5b-4ee2-9494-71a9801a17ad admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:08.671126 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85326e4c-bb5b-4ee2-9494-71a9801a17ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:08.671399 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85326e4c-bb5b-4ee2-9494-71a9801a17ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:08.672071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85326e4c-bb5b-4ee2-9494-71a9801a17ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:06:08.672377 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85326e4c-bb5b-4ee2-9494-71a9801a17ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:08.673978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85326e4c-bb5b-4ee2-9494-71a9801a17ad admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:08.687530 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1346/2684] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:06:08.693238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee915a-8635-44c3-b260-ecef08ae4c6e None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:08.702924 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee915a-8635-44c3-b260-ecef08ae4c6e None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:08.707514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40613e72-a3c0-445a-a270-f3da3a58d08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:08.708119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40613e72-a3c0-445a-a270-f3da3a58d08a admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:08.708398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40613e72-a3c0-445a-a270-f3da3a58d08a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:08.708924 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40613e72-a3c0-445a-a270-f3da3a58d08a admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:08.709548 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40613e72-a3c0-445a-a270-f3da3a58d08a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:08.711498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40613e72-a3c0-445a-a270-f3da3a58d08a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:08.748811 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1339/2685] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:08 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:08.748928 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85326e4c-bb5b-4ee2-9494-71a9801a17ad admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:08.760097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85326e4c-bb5b-4ee2-9494-71a9801a17ad admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:08.816207 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbab98-0b13-4479-a494-8e4a7585ebd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:08.816831 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbab98-0b13-4479-a494-8e4a7585ebd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:08.817107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbab98-0b13-4479-a494-8e4a7585ebd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:08.817397 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbab98-0b13-4479-a494-8e4a7585ebd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:08.818046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbab98-0b13-4479-a494-8e4a7585ebd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:08.819973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbab98-0b13-4479-a494-8e4a7585ebd0 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:08.841289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1347/2686] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:08 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:08.845462 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40613e72-a3c0-445a-a270-f3da3a58d08a admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:08.861746 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40613e72-a3c0-445a-a270-f3da3a58d08a admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:08.866443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edaaa5b-d2f9-4a28-a6bc-5ffe0343d073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:08.867080 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edaaa5b-d2f9-4a28-a6bc-5ffe0343d073 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:08.867366 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edaaa5b-d2f9-4a28-a6bc-5ffe0343d073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:08.867637 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edaaa5b-d2f9-4a28-a6bc-5ffe0343d073 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:08.868277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edaaa5b-d2f9-4a28-a6bc-5ffe0343d073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:08.869924 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edaaa5b-d2f9-4a28-a6bc-5ffe0343d073 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:08.879987 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1340/2687] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:08.884055 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbab98-0b13-4479-a494-8e4a7585ebd0 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:08.893945 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbab98-0b13-4479-a494-8e4a7585ebd0 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:08.898508 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770feb2b-1df6-490c-9a74-ff9b21e61fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:08.899122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770feb2b-1df6-490c-9a74-ff9b21e61fad admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:08.899398 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770feb2b-1df6-490c-9a74-ff9b21e61fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:08.899671 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770feb2b-1df6-490c-9a74-ff9b21e61fad admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:08.900298 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770feb2b-1df6-490c-9a74-ff9b21e61fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:08.902417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770feb2b-1df6-490c-9a74-ff9b21e61fad admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:08.913310 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1348/2688] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:08.918315 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edaaa5b-d2f9-4a28-a6bc-5ffe0343d073 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:08.928549 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edaaa5b-d2f9-4a28-a6bc-5ffe0343d073 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:08.933367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8afc4b-accf-479a-98fa-c661b180ea76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:08.934011 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8afc4b-accf-479a-98fa-c661b180ea76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:08.934287 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8afc4b-accf-479a-98fa-c661b180ea76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:08.934554 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8afc4b-accf-479a-98fa-c661b180ea76 admin admin] PATH_INFO: `/v3/projects/e5817c5826f145a4a39bb0e28c9b847d/users/0a5ae27daa714d79943c4ec397158ddf/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:08.948818 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8afc4b-accf-479a-98fa-c661b180ea76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5817c5826f145a4a39bb0e28c9b847d, user_id=0a5ae27daa714d79943c4ec397158ddf, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:08.951544 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8afc4b-accf-479a-98fa-c661b180ea76 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:08.964544 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1341/2689] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:08 2022] PUT /identity/v3/projects/e5817c5826f145a4a39bb0e28c9b847d/users/0a5ae27daa714d79943c4ec397158ddf/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:08.975842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414cbd6-a169-45d2-a3f0-ee91191a5d2d None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:08.976330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414cbd6-a169-45d2-a3f0-ee91191a5d2d None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:08.976463 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414cbd6-a169-45d2-a3f0-ee91191a5d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:09.004275 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-3414cbd6-a169-45d2-a3f0-ee91191a5d2d None None] MFA Rules not processed for user `0a5ae27daa714d79943c4ec397158ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:09.006123 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3414cbd6-a169-45d2-a3f0-ee91191a5d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:09.045135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:09.073158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1349/2690] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:09.077798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8afc4b-accf-479a-98fa-c661b180ea76 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:09.105834 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:09.143213 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8afc4b-accf-479a-98fa-c661b180ea76 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:09.147926 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa2f51-42b7-4e84-aec7-1930e2b1695f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:09.148561 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa2f51-42b7-4e84-aec7-1930e2b1695f admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:09.148835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa2f51-42b7-4e84-aec7-1930e2b1695f admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:09.149104 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa2f51-42b7-4e84-aec7-1930e2b1695f admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:09.149742 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa2f51-42b7-4e84-aec7-1930e2b1695f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:09.152157 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa2f51-42b7-4e84-aec7-1930e2b1695f admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:09.178558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1342/2691] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:09 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:09.182867 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414cbd6-a169-45d2-a3f0-ee91191a5d2d None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:09.193524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414cbd6-a169-45d2-a3f0-ee91191a5d2d None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:09.250645 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffa5b9-b190-4e7f-9dbf-07dd72881832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:09.251278 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffa5b9-b190-4e7f-9dbf-07dd72881832 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:09.251560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffa5b9-b190-4e7f-9dbf-07dd72881832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:09.251832 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffa5b9-b190-4e7f-9dbf-07dd72881832 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:09.252605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffa5b9-b190-4e7f-9dbf-07dd72881832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:09.254384 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffa5b9-b190-4e7f-9dbf-07dd72881832 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:09.275590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1350/2692] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:09 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:09.280211 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa2f51-42b7-4e84-aec7-1930e2b1695f admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:09.290443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa2f51-42b7-4e84-aec7-1930e2b1695f admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:09.295125 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c496d34-8f33-4bf7-a2b1-9f38b86c1bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:09.295720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c496d34-8f33-4bf7-a2b1-9f38b86c1bde admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:09.296000 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c496d34-8f33-4bf7-a2b1-9f38b86c1bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:09.296277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c496d34-8f33-4bf7-a2b1-9f38b86c1bde admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:09.296913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c496d34-8f33-4bf7-a2b1-9f38b86c1bde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:09.298492 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c496d34-8f33-4bf7-a2b1-9f38b86c1bde admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:09.309432 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1343/2693] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:09.313502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffa5b9-b190-4e7f-9dbf-07dd72881832 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:09.324046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffa5b9-b190-4e7f-9dbf-07dd72881832 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:09.328713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f94d5-7b2b-4981-8a85-20b3e3b1a6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:09.329378 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f94d5-7b2b-4981-8a85-20b3e3b1a6dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:09.329702 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f94d5-7b2b-4981-8a85-20b3e3b1a6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:09.329911 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f94d5-7b2b-4981-8a85-20b3e3b1a6dd admin admin] PATH_INFO: `/v3/projects/13a61082943f4f279bc092600badc63e/users/02843d7152dc4aeaa9d06c0baa0666bb/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:09.346009 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f94d5-7b2b-4981-8a85-20b3e3b1a6dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=13a61082943f4f279bc092600badc63e, user_id=02843d7152dc4aeaa9d06c0baa0666bb, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:09.348692 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f94d5-7b2b-4981-8a85-20b3e3b1a6dd admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:09.363621 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1351/2694] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:09 2022] PUT /identity/v3/projects/13a61082943f4f279bc092600badc63e/users/02843d7152dc4aeaa9d06c0baa0666bb/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:09.367479 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c496d34-8f33-4bf7-a2b1-9f38b86c1bde admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:09.386360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c496d34-8f33-4bf7-a2b1-9f38b86c1bde admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:09.438129 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100ffe7-5092-4645-8880-8bf36842e1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:09.438871 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100ffe7-5092-4645-8880-8bf36842e1dd admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:09.439152 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100ffe7-5092-4645-8880-8bf36842e1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:09.439442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100ffe7-5092-4645-8880-8bf36842e1dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:09.440096 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100ffe7-5092-4645-8880-8bf36842e1dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:09.441717 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100ffe7-5092-4645-8880-8bf36842e1dd admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:09.452197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1344/2695] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:09.456750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f94d5-7b2b-4981-8a85-20b3e3b1a6dd admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:09.468132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f94d5-7b2b-4981-8a85-20b3e3b1a6dd admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:09.472973 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3954f5-e586-4d23-84e9-cca69117c4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:09.473628 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3954f5-e586-4d23-84e9-cca69117c4e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:09.473905 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3954f5-e586-4d23-84e9-cca69117c4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:09.475178 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3954f5-e586-4d23-84e9-cca69117c4e7 admin admin] PATH_INFO: `/v3/projects/13a61082943f4f279bc092600badc63e/users/02843d7152dc4aeaa9d06c0baa0666bb/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:09.476133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3954f5-e586-4d23-84e9-cca69117c4e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13a61082943f4f279bc092600badc63e, user_id=02843d7152dc4aeaa9d06c0baa0666bb, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:09.478770 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3954f5-e586-4d23-84e9-cca69117c4e7 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:09.489583 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1352/2696] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:09 2022] PUT /identity/v3/projects/13a61082943f4f279bc092600badc63e/users/02843d7152dc4aeaa9d06c0baa0666bb/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:09.499846 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85cde9-78b4-44f4-924f-1ad9f28ecd01 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:09.500136 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85cde9-78b4-44f4-924f-1ad9f28ecd01 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:09.500645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85cde9-78b4-44f4-924f-1ad9f28ecd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:09.529953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-be85cde9-78b4-44f4-924f-1ad9f28ecd01 None None] MFA Rules not processed for user `02843d7152dc4aeaa9d06c0baa0666bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:09.531198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-be85cde9-78b4-44f4-924f-1ad9f28ecd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:09.606241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1345/2697] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:09.613613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3954f5-e586-4d23-84e9-cca69117c4e7 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:09.615402 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-da3954f5-e586-4d23-84e9-cca69117c4e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:09.677890 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3954f5-e586-4d23-84e9-cca69117c4e7 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:09.681613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac8061-c587-4f9b-8bf1-8036c8177eb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:09.682221 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac8061-c587-4f9b-8bf1-8036c8177eb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:09.682499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac8061-c587-4f9b-8bf1-8036c8177eb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:09.683282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac8061-c587-4f9b-8bf1-8036c8177eb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:09.692442 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac8061-c587-4f9b-8bf1-8036c8177eb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:09.693867 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac8061-c587-4f9b-8bf1-8036c8177eb1 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:09.704562 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1353/2698] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:09.808679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85cde9-78b4-44f4-924f-1ad9f28ecd01 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:09.818468 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85cde9-78b4-44f4-924f-1ad9f28ecd01 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:09.822148 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d48a2-4028-4430-9410-8c06bb4ee749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:09.822791 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d48a2-4028-4430-9410-8c06bb4ee749 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:09.823075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d48a2-4028-4430-9410-8c06bb4ee749 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:09.824004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d48a2-4028-4430-9410-8c06bb4ee749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:09.837678 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:09.890127 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d48a2-4028-4430-9410-8c06bb4ee749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:09.891505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d48a2-4028-4430-9410-8c06bb4ee749 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:09.900442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1346/2699] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:06:12.405071 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac8061-c587-4f9b-8bf1-8036c8177eb1 service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:12.406506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-70ac8061-c587-4f9b-8bf1-8036c8177eb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:12.444685 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:12.481937 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac8061-c587-4f9b-8bf1-8036c8177eb1 service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:12.487412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b74e6-8bef-4720-92cc-dab1276085d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:12.488140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b74e6-8bef-4720-92cc-dab1276085d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:12.488495 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b74e6-8bef-4720-92cc-dab1276085d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:12.488844 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b74e6-8bef-4720-92cc-dab1276085d9 admin admin] PATH_INFO: `/v3/users/65d658e7417c422c8700a0435f0caba2` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:12.490062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b74e6-8bef-4720-92cc-dab1276085d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65d658e7417c422c8700a0435f0caba2)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:12.492096 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b74e6-8bef-4720-92cc-dab1276085d9 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:12.504669 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d48a2-4028-4430-9410-8c06bb4ee749 service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:12.520740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d48a2-4028-4430-9410-8c06bb4ee749 service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:12.527156 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc38b72-cb5f-4e77-9864-6f3260a108cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:12.527811 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc38b72-cb5f-4e77-9864-6f3260a108cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:12.528116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc38b72-cb5f-4e77-9864-6f3260a108cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:12.528360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc38b72-cb5f-4e77-9864-6f3260a108cc admin admin] PATH_INFO: `/v3/users/0a5ae27daa714d79943c4ec397158ddf` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:12.529607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc38b72-cb5f-4e77-9864-6f3260a108cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a5ae27daa714d79943c4ec397158ddf)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:12.533168 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc38b72-cb5f-4e77-9864-6f3260a108cc admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:12.565009 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a05b74e6-8bef-4720-92cc-dab1276085d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65d658e7417c422c8700a0435f0caba2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:12.576584 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a05b74e6-8bef-4720-92cc-dab1276085d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65d658e7417c422c8700a0435f0caba2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:12.588133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a05b74e6-8bef-4720-92cc-dab1276085d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65d658e7417c422c8700a0435f0caba2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:12.608845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a05b74e6-8bef-4720-92cc-dab1276085d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65d658e7417c422c8700a0435f0caba2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:12.609677 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a05b74e6-8bef-4720-92cc-dab1276085d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65d658e7417c422c8700a0435f0caba2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:12.621751 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1354/2700] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:12 2022] DELETE /identity/v3/users/65d658e7417c422c8700a0435f0caba2 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:12.621995 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6bc38b72-cb5f-4e77-9864-6f3260a108cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a5ae27daa714d79943c4ec397158ddf'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:12.624074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee9470-3502-4941-85cf-1eaf15291065 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:12.624436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee9470-3502-4941-85cf-1eaf15291065 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:12.624962 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee9470-3502-4941-85cf-1eaf15291065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:12.630325 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6bc38b72-cb5f-4e77-9864-6f3260a108cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a5ae27daa714d79943c4ec397158ddf'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:12.631428 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6bc38b72-cb5f-4e77-9864-6f3260a108cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a5ae27daa714d79943c4ec397158ddf'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:12.649825 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6bc38b72-cb5f-4e77-9864-6f3260a108cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a5ae27daa714d79943c4ec397158ddf'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:12.651051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-6bc38b72-cb5f-4e77-9864-6f3260a108cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a5ae27daa714d79943c4ec397158ddf'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:12.654946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-1eee9470-3502-4941-85cf-1eaf15291065 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:12.656838 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1eee9470-3502-4941-85cf-1eaf15291065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:12.666341 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1347/2701] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:12 2022] DELETE /identity/v3/users/0a5ae27daa714d79943c4ec397158ddf => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:12.671361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdeb78-e42f-46f4-9fea-e7ede0a7ca3a None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:12.671730 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdeb78-e42f-46f4-9fea-e7ede0a7ca3a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:12.672180 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cdeb78-e42f-46f4-9fea-e7ede0a7ca3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:12.697269 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-e1cdeb78-e42f-46f4-9fea-e7ede0a7ca3a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:12.699228 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e1cdeb78-e42f-46f4-9fea-e7ede0a7ca3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:12.743880 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1355/2702] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:12.762569 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee9470-3502-4941-85cf-1eaf15291065 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:12.765343 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1eee9470-3502-4941-85cf-1eaf15291065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:12.796522 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1348/2703] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:12.812126 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdeb78-e42f-46f4-9fea-e7ede0a7ca3a None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:12.813760 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e1cdeb78-e42f-46f4-9fea-e7ede0a7ca3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:12.857079 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:12.863371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee9470-3502-4941-85cf-1eaf15291065 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:12.869149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:12.902442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cdeb78-e42f-46f4-9fea-e7ede0a7ca3a None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:12.906724 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f933335-42fc-4ea4-9a30-ad390217894c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:12.907602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f933335-42fc-4ea4-9a30-ad390217894c service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:12.907880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f933335-42fc-4ea4-9a30-ad390217894c service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:12.908153 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f933335-42fc-4ea4-9a30-ad390217894c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:12.918448 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f933335-42fc-4ea4-9a30-ad390217894c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:12.919993 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f933335-42fc-4ea4-9a30-ad390217894c service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:12.931027 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1349/2704] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:12.971140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f90a86-9cc1-4bf4-99e7-0f7844e3e6f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:12.972000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f90a86-9cc1-4bf4-99e7-0f7844e3e6f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:12.972000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f90a86-9cc1-4bf4-99e7-0f7844e3e6f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:12.972000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f90a86-9cc1-4bf4-99e7-0f7844e3e6f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:12.989227 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f90a86-9cc1-4bf4-99e7-0f7844e3e6f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:12.991597 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f90a86-9cc1-4bf4-99e7-0f7844e3e6f8 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:13.007941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1356/2705] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:13.326073 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f933335-42fc-4ea4-9a30-ad390217894c service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:13.331431 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-0f933335-42fc-4ea4-9a30-ad390217894c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:13.356624 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f933335-42fc-4ea4-9a30-ad390217894c service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:13.362536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:13.363269 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:13.363549 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:13.363817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27 admin admin] PATH_INFO: `/v3/projects/e5817c5826f145a4a39bb0e28c9b847d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:13.364990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5817c5826f145a4a39bb0e28c9b847d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:13.367120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:13.393664 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f90a86-9cc1-4bf4-99e7-0f7844e3e6f8 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:13.395376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-c9f90a86-9cc1-4bf4-99e7-0f7844e3e6f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:13.421182 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f90a86-9cc1-4bf4-99e7-0f7844e3e6f8 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:13.427442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5817c5826f145a4a39bb0e28c9b847d', 'request_id': 'req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.438055 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5817c5826f145a4a39bb0e28c9b847d', 'request_id': 'req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.447376 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5817c5826f145a4a39bb0e28c9b847d', 'request_id': 'req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.459587 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27 admin admin] The token cache is being invalidate because project e5817c5826f145a4a39bb0e28c9b847d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:13.459881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.460549 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.462105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1350/2706] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:13 2022] DELETE /identity/v3/projects/e5817c5826f145a4a39bb0e28c9b847d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:13.466704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:13.468194 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:13.502101 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68c220-b9cc-4022-8772-28b6dc80aef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:13.502796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68c220-b9cc-4022-8772-28b6dc80aef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:13.503077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68c220-b9cc-4022-8772-28b6dc80aef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:13.503353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68c220-b9cc-4022-8772-28b6dc80aef7 admin admin] PATH_INFO: `/v3/projects/fe05a44283164990b3f411e8e2f19b4c` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:13.505014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68c220-b9cc-4022-8772-28b6dc80aef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe05a44283164990b3f411e8e2f19b4c)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:13.507075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68c220-b9cc-4022-8772-28b6dc80aef7 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:13.540552 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0ea43-7f40-4b82-b72b-ac6d3fe29e27 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:13.546440 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee55131-0047-47ee-b092-bb49e0337696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:13.547119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee55131-0047-47ee-b092-bb49e0337696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:13.547398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee55131-0047-47ee-b092-bb49e0337696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:13.547670 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee55131-0047-47ee-b092-bb49e0337696 admin admin] PATH_INFO: `/v3/users/02843d7152dc4aeaa9d06c0baa0666bb` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:13.548809 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee55131-0047-47ee-b092-bb49e0337696 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02843d7152dc4aeaa9d06c0baa0666bb)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:13.550753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee55131-0047-47ee-b092-bb49e0337696 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:13.563414 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3b68c220-b9cc-4022-8772-28b6dc80aef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe05a44283164990b3f411e8e2f19b4c', 'request_id': 'req-3b68c220-b9cc-4022-8772-28b6dc80aef7'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.569526 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3b68c220-b9cc-4022-8772-28b6dc80aef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe05a44283164990b3f411e8e2f19b4c', 'request_id': 'req-3b68c220-b9cc-4022-8772-28b6dc80aef7'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.575156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3b68c220-b9cc-4022-8772-28b6dc80aef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe05a44283164990b3f411e8e2f19b4c', 'request_id': 'req-3b68c220-b9cc-4022-8772-28b6dc80aef7'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.588546 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3b68c220-b9cc-4022-8772-28b6dc80aef7 admin admin] The token cache is being invalidate because project fe05a44283164990b3f411e8e2f19b4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:13.589245 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3b68c220-b9cc-4022-8772-28b6dc80aef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.590024 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-3b68c220-b9cc-4022-8772-28b6dc80aef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.592550 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1357/2707] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:13 2022] DELETE /identity/v3/projects/fe05a44283164990b3f411e8e2f19b4c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:13.596813 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68c220-b9cc-4022-8772-28b6dc80aef7 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:13.598215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3b68c220-b9cc-4022-8772-28b6dc80aef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:13.601833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bee55131-0047-47ee-b092-bb49e0337696 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02843d7152dc4aeaa9d06c0baa0666bb'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.610334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bee55131-0047-47ee-b092-bb49e0337696 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02843d7152dc4aeaa9d06c0baa0666bb'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.611125 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bee55131-0047-47ee-b092-bb49e0337696 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02843d7152dc4aeaa9d06c0baa0666bb'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.613371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:13.625961 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bee55131-0047-47ee-b092-bb49e0337696 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02843d7152dc4aeaa9d06c0baa0666bb'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.626815 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-bee55131-0047-47ee-b092-bb49e0337696 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02843d7152dc4aeaa9d06c0baa0666bb'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.641579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1351/2708] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:13 2022] DELETE /identity/v3/users/02843d7152dc4aeaa9d06c0baa0666bb => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:13.646129 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:13.668747 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68c220-b9cc-4022-8772-28b6dc80aef7 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:13.729727 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d39f1-9115-472c-a4af-1472d37bcf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:13.730368 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d39f1-9115-472c-a4af-1472d37bcf97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:13.730672 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d39f1-9115-472c-a4af-1472d37bcf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:13.730988 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d39f1-9115-472c-a4af-1472d37bcf97 admin admin] PATH_INFO: `/v3/users/69bcec515d484037ab8afdc6defa067a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:13.732166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d39f1-9115-472c-a4af-1472d37bcf97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69bcec515d484037ab8afdc6defa067a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:13.734313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d39f1-9115-472c-a4af-1472d37bcf97 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:13.783999 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-381d39f1-9115-472c-a4af-1472d37bcf97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69bcec515d484037ab8afdc6defa067a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.792452 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-381d39f1-9115-472c-a4af-1472d37bcf97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69bcec515d484037ab8afdc6defa067a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.804912 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-381d39f1-9115-472c-a4af-1472d37bcf97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69bcec515d484037ab8afdc6defa067a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.819633 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-381d39f1-9115-472c-a4af-1472d37bcf97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69bcec515d484037ab8afdc6defa067a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.820501 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-381d39f1-9115-472c-a4af-1472d37bcf97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69bcec515d484037ab8afdc6defa067a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:13.826707 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1358/2709] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:13 2022] DELETE /identity/v3/users/69bcec515d484037ab8afdc6defa067a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:06:13.950507 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee55131-0047-47ee-b092-bb49e0337696 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:13.953569 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-bee55131-0047-47ee-b092-bb49e0337696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:13.985601 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee55131-0047-47ee-b092-bb49e0337696 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:14.050535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72386c21-97ff-4c0f-b7df-666a67b854c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:14.051128 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72386c21-97ff-4c0f-b7df-666a67b854c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.051410 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72386c21-97ff-4c0f-b7df-666a67b854c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.051687 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72386c21-97ff-4c0f-b7df-666a67b854c5 admin admin] PATH_INFO: `/v3/projects/13a61082943f4f279bc092600badc63e` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.052841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72386c21-97ff-4c0f-b7df-666a67b854c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13a61082943f4f279bc092600badc63e)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:14.054842 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72386c21-97ff-4c0f-b7df-666a67b854c5 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:14.108034 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-72386c21-97ff-4c0f-b7df-666a67b854c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13a61082943f4f279bc092600badc63e', 'request_id': 'req-72386c21-97ff-4c0f-b7df-666a67b854c5'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:14.115516 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-72386c21-97ff-4c0f-b7df-666a67b854c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13a61082943f4f279bc092600badc63e', 'request_id': 'req-72386c21-97ff-4c0f-b7df-666a67b854c5'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:14.122339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-72386c21-97ff-4c0f-b7df-666a67b854c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13a61082943f4f279bc092600badc63e', 'request_id': 'req-72386c21-97ff-4c0f-b7df-666a67b854c5'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:14.133928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-72386c21-97ff-4c0f-b7df-666a67b854c5 admin admin] The token cache is being invalidate because project 13a61082943f4f279bc092600badc63e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:14.134210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-72386c21-97ff-4c0f-b7df-666a67b854c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:14.134917 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-72386c21-97ff-4c0f-b7df-666a67b854c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:14.136655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1352/2710] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:13 2022] DELETE /identity/v3/projects/13a61082943f4f279bc092600badc63e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:14.141845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140da15-7483-4580-9ccf-08fc73823212 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.142132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140da15-7483-4580-9ccf-08fc73823212 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.142484 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140da15-7483-4580-9ccf-08fc73823212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.147620 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72386c21-97ff-4c0f-b7df-666a67b854c5 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:14.149172 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-72386c21-97ff-4c0f-b7df-666a67b854c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:14.165412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-1140da15-7483-4580-9ccf-08fc73823212 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:14.166584 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1140da15-7483-4580-9ccf-08fc73823212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:14.218668 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72386c21-97ff-4c0f-b7df-666a67b854c5 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:14.224025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d7bb8-330c-4d7b-80d5-3b7f0b572e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:14.224648 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d7bb8-330c-4d7b-80d5-3b7f0b572e74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.224930 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d7bb8-330c-4d7b-80d5-3b7f0b572e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.225202 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d7bb8-330c-4d7b-80d5-3b7f0b572e74 admin admin] PATH_INFO: `/v3/projects/ab1087e94a8e4736b1a8e612ec5616fe` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.226330 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216d7bb8-330c-4d7b-80d5-3b7f0b572e74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab1087e94a8e4736b1a8e612ec5616fe)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:14.228261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216d7bb8-330c-4d7b-80d5-3b7f0b572e74 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:14.228533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1359/2711] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:14.236253 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:14.236880 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f8e0c-842c-4269-b300-63c0d9d16650 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.237185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f8e0c-842c-4269-b300-63c0d9d16650 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.237494 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f8e0c-842c-4269-b300-63c0d9d16650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.257379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-479f8e0c-842c-4269-b300-63c0d9d16650 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:14.258582 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-479f8e0c-842c-4269-b300-63c0d9d16650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:14.267513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1360/2712] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:06:14.271458 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f8e0c-842c-4269-b300-63c0d9d16650 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:14.275676 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:14.281499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f8e0c-842c-4269-b300-63c0d9d16650 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:14.281680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-216d7bb8-330c-4d7b-80d5-3b7f0b572e74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab1087e94a8e4736b1a8e612ec5616fe', 'request_id': 'req-216d7bb8-330c-4d7b-80d5-3b7f0b572e74'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:14.289856 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-216d7bb8-330c-4d7b-80d5-3b7f0b572e74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab1087e94a8e4736b1a8e612ec5616fe', 'request_id': 'req-216d7bb8-330c-4d7b-80d5-3b7f0b572e74'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:14.296858 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-216d7bb8-330c-4d7b-80d5-3b7f0b572e74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab1087e94a8e4736b1a8e612ec5616fe', 'request_id': 'req-216d7bb8-330c-4d7b-80d5-3b7f0b572e74'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:14.308048 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-216d7bb8-330c-4d7b-80d5-3b7f0b572e74 admin admin] The token cache is being invalidate because project ab1087e94a8e4736b1a8e612ec5616fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:14.308299 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-216d7bb8-330c-4d7b-80d5-3b7f0b572e74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:14.308938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-216d7bb8-330c-4d7b-80d5-3b7f0b572e74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:14.313881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1353/2713] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:14 2022] DELETE /identity/v3/projects/ab1087e94a8e4736b1a8e612ec5616fe => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:06:14.315455 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998497d-482b-4dae-8157-27335645cf7a None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.315757 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998497d-482b-4dae-8157-27335645cf7a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.316021 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998497d-482b-4dae-8157-27335645cf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.336565 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-1998497d-482b-4dae-8157-27335645cf7a None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:14.337786 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-1998497d-482b-4dae-8157-27335645cf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:14.351978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d36af-e74e-42d2-9f85-cc3b97b3b81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:14.352599 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d36af-e74e-42d2-9f85-cc3b97b3b81e admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.352875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d36af-e74e-42d2-9f85-cc3b97b3b81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.353150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d36af-e74e-42d2-9f85-cc3b97b3b81e admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.353788 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d36af-e74e-42d2-9f85-cc3b97b3b81e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:06:14.354089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d36af-e74e-42d2-9f85-cc3b97b3b81e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:14.356287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d36af-e74e-42d2-9f85-cc3b97b3b81e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:14.373180 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1361/2714] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 15:06:14.374168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d36af-e74e-42d2-9f85-cc3b97b3b81e admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:14.375862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d85d36af-e74e-42d2-9f85-cc3b97b3b81e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:14.388234 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d36af-e74e-42d2-9f85-cc3b97b3b81e admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:14.392936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423ddb8-dd2c-43c0-b489-ea138927933e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:14.393533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423ddb8-dd2c-43c0-b489-ea138927933e admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.393816 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423ddb8-dd2c-43c0-b489-ea138927933e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.394089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423ddb8-dd2c-43c0-b489-ea138927933e admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.394720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9423ddb8-dd2c-43c0-b489-ea138927933e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:14.396684 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9423ddb8-dd2c-43c0-b489-ea138927933e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:14.403621 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1354/2715] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:06:14.411416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9362c42-59b9-494a-bbdd-528fb0a22b48 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.411699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9362c42-59b9-494a-bbdd-528fb0a22b48 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.412099 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9362c42-59b9-494a-bbdd-528fb0a22b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.429097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1362/2716] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:14 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:14.430718 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423ddb8-dd2c-43c0-b489-ea138927933e admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:14.434679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-c9362c42-59b9-494a-bbdd-528fb0a22b48 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:14.436115 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-c9362c42-59b9-494a-bbdd-528fb0a22b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:14.497921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423ddb8-dd2c-43c0-b489-ea138927933e admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:14.500293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1355/2717] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:06:14.502610 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f65ed-5b36-4cb3-9dd3-91dac8e22ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:14.503251 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f65ed-5b36-4cb3-9dd3-91dac8e22ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.503527 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f65ed-5b36-4cb3-9dd3-91dac8e22ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.503799 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f65ed-5b36-4cb3-9dd3-91dac8e22ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.504420 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f65ed-5b36-4cb3-9dd3-91dac8e22ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:14.504606 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9362c42-59b9-494a-bbdd-528fb0a22b48 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:14.506331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f65ed-5b36-4cb3-9dd3-91dac8e22ad5 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:14.514140 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9362c42-59b9-494a-bbdd-528fb0a22b48 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:14.518791 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bae97-65c9-46e1-ad78-78cbb52c994e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:14.519427 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bae97-65c9-46e1-ad78-78cbb52c994e admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.519685 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bae97-65c9-46e1-ad78-78cbb52c994e admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.520076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bae97-65c9-46e1-ad78-78cbb52c994e admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.520873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9bae97-65c9-46e1-ad78-78cbb52c994e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:06:14.521273 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9bae97-65c9-46e1-ad78-78cbb52c994e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:14.523054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9bae97-65c9-46e1-ad78-78cbb52c994e admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:14.529120 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1363/2718] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:14 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:14.533102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f65ed-5b36-4cb3-9dd3-91dac8e22ad5 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:14.536191 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1356/2719] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 15:06:14.540810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bae97-65c9-46e1-ad78-78cbb52c994e admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:14.544602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f65ed-5b36-4cb3-9dd3-91dac8e22ad5 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:14.549726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf631be6-53c7-44fa-aebf-0f56aa300347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:14.550468 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf631be6-53c7-44fa-aebf-0f56aa300347 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.550759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf631be6-53c7-44fa-aebf-0f56aa300347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.551048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf631be6-53c7-44fa-aebf-0f56aa300347 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.551696 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf631be6-53c7-44fa-aebf-0f56aa300347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:14.553638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf631be6-53c7-44fa-aebf-0f56aa300347 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:14.560282 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bae97-65c9-46e1-ad78-78cbb52c994e admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:14.565966 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1364/2720] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:14.566147 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bef1e4-dba3-4c89-9632-67c7c2fdd5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:14.566682 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bef1e4-dba3-4c89-9632-67c7c2fdd5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.567340 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bef1e4-dba3-4c89-9632-67c7c2fdd5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.567623 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bef1e4-dba3-4c89-9632-67c7c2fdd5b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.569652 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf631be6-53c7-44fa-aebf-0f56aa300347 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:14.569947 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bef1e4-dba3-4c89-9632-67c7c2fdd5b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:14.571996 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bef1e4-dba3-4c89-9632-67c7c2fdd5b1 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:14.581689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf631be6-53c7-44fa-aebf-0f56aa300347 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:14.587353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612726af-63e3-4107-a751-4a790ed81326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:14.588203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612726af-63e3-4107-a751-4a790ed81326 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.589528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612726af-63e3-4107-a751-4a790ed81326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.590007 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612726af-63e3-4107-a751-4a790ed81326 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.591102 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612726af-63e3-4107-a751-4a790ed81326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:14.593056 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612726af-63e3-4107-a751-4a790ed81326 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:14.606796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1365/2721] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:06:14.606923 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:14.611312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612726af-63e3-4107-a751-4a790ed81326 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:14.620325 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1357/2722] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:14 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:14.624519 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bef1e4-dba3-4c89-9632-67c7c2fdd5b1 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:14.627210 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612726af-63e3-4107-a751-4a790ed81326 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:14.682235 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:14.716248 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c6c7d-130b-412e-b94f-bf51b315ae48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:14.717475 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c6c7d-130b-412e-b94f-bf51b315ae48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.718355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c6c7d-130b-412e-b94f-bf51b315ae48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.718873 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c6c7d-130b-412e-b94f-bf51b315ae48 admin admin] PATH_INFO: `/v3/projects/f3c4548dcad04f1b971ed0c24ec3122e/users/5e7557cf9d3f4951846386b91197b7ee/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.731242 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bef1e4-dba3-4c89-9632-67c7c2fdd5b1 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:14.737713 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44168d-4f5f-4af9-ad53-3d9c958c6b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:14.738537 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44168d-4f5f-4af9-ad53-3d9c958c6b28 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.738998 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44168d-4f5f-4af9-ad53-3d9c958c6b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.739325 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44168d-4f5f-4af9-ad53-3d9c958c6b28 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.740296 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44168d-4f5f-4af9-ad53-3d9c958c6b28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:14.742523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44168d-4f5f-4af9-ad53-3d9c958c6b28 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:14.743646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c6c7d-130b-412e-b94f-bf51b315ae48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3c4548dcad04f1b971ed0c24ec3122e, user_id=5e7557cf9d3f4951846386b91197b7ee, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:14.747023 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c6c7d-130b-412e-b94f-bf51b315ae48 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:14.793781 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1366/2723] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:14 2022] PUT /identity/v3/projects/f3c4548dcad04f1b971ed0c24ec3122e/users/5e7557cf9d3f4951846386b91197b7ee/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:06:14.793989 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1358/2724] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:14 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:14.794093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c6c7d-130b-412e-b94f-bf51b315ae48 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:14.796660 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ab2ba-26e2-492e-9062-b1421de58a4a None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.797005 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ab2ba-26e2-492e-9062-b1421de58a4a None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.797343 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ab2ba-26e2-492e-9062-b1421de58a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.829466 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-507ab2ba-26e2-492e-9062-b1421de58a4a None None] MFA Rules not processed for user `5e7557cf9d3f4951846386b91197b7ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:14.830951 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-507ab2ba-26e2-492e-9062-b1421de58a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:14.863677 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c6c7d-130b-412e-b94f-bf51b315ae48 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:14.868625 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc8c52-1384-4ec1-b917-2b100762691e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:14.869637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc8c52-1384-4ec1-b917-2b100762691e admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.870565 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc8c52-1384-4ec1-b917-2b100762691e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.870991 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc8c52-1384-4ec1-b917-2b100762691e admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.871747 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc8c52-1384-4ec1-b917-2b100762691e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:14.873968 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc8c52-1384-4ec1-b917-2b100762691e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:14.886254 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1367/2725] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:14.890364 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc8c52-1384-4ec1-b917-2b100762691e admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:14.902534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc8c52-1384-4ec1-b917-2b100762691e admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:14.908006 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67ce09-17fd-4b2b-a617-6f5cc2113cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:14.908821 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67ce09-17fd-4b2b-a617-6f5cc2113cca admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.909170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67ce09-17fd-4b2b-a617-6f5cc2113cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.909533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67ce09-17fd-4b2b-a617-6f5cc2113cca admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.910264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67ce09-17fd-4b2b-a617-6f5cc2113cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:14.911978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67ce09-17fd-4b2b-a617-6f5cc2113cca admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:14.916877 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1359/2726] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:14.921443 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ab2ba-26e2-492e-9062-b1421de58a4a None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:14.926897 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1368/2727] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:06:14.928370 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67ce09-17fd-4b2b-a617-6f5cc2113cca admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:14.933427 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ab2ba-26e2-492e-9062-b1421de58a4a None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:14.939764 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef5a1c-d797-40a6-bd06-b54f8b215ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:14.940112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67ce09-17fd-4b2b-a617-6f5cc2113cca admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:14.940352 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef5a1c-d797-40a6-bd06-b54f8b215ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.940627 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef5a1c-d797-40a6-bd06-b54f8b215ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.940899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef5a1c-d797-40a6-bd06-b54f8b215ec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.941530 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef5a1c-d797-40a6-bd06-b54f8b215ec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:14.943514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef5a1c-d797-40a6-bd06-b54f8b215ec9 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:14.945004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d89cd3-0451-4de6-8249-8afe0395d601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:14.945737 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d89cd3-0451-4de6-8249-8afe0395d601 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.946021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d89cd3-0451-4de6-8249-8afe0395d601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.946295 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d89cd3-0451-4de6-8249-8afe0395d601 admin admin] PATH_INFO: `/v3/projects/cf3a11b11545423f8cc9e235e4acd31e/users/5a3883f7400d4d51a3f4459ce3a79b56/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:14.965294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d89cd3-0451-4de6-8249-8afe0395d601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf3a11b11545423f8cc9e235e4acd31e, user_id=5a3883f7400d4d51a3f4459ce3a79b56, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:14.968158 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d89cd3-0451-4de6-8249-8afe0395d601 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:14.973035 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1360/2728] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:14 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:14.976982 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef5a1c-d797-40a6-bd06-b54f8b215ec9 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:14.986050 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1369/2729] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:14 2022] PUT /identity/v3/projects/cf3a11b11545423f8cc9e235e4acd31e/users/5a3883f7400d4d51a3f4459ce3a79b56/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:14.998116 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ec943-e284-41ed-986b-32844d440ee5 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:14.998400 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ec943-e284-41ed-986b-32844d440ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:14.998682 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ec943-e284-41ed-986b-32844d440ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:15.027744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-6c3ec943-e284-41ed-986b-32844d440ee5 None None] MFA Rules not processed for user `5a3883f7400d4d51a3f4459ce3a79b56`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:15.029280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-6c3ec943-e284-41ed-986b-32844d440ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:15.052028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:15.056262 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef5a1c-d797-40a6-bd06-b54f8b215ec9 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:15.105715 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1370/2730] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:06:15.110060 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ec943-e284-41ed-986b-32844d440ee5 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:15.126526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268871c3-cb59-48ee-9198-3309b144ac29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:15.127253 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268871c3-cb59-48ee-9198-3309b144ac29 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:15.127533 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268871c3-cb59-48ee-9198-3309b144ac29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:15.127857 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268871c3-cb59-48ee-9198-3309b144ac29 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:15.128447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268871c3-cb59-48ee-9198-3309b144ac29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:15.130360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268871c3-cb59-48ee-9198-3309b144ac29 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:15.139180 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:15.153705 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1361/2731] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:14 2022] POST /identity/v3/users => generated 299 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:15.157780 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268871c3-cb59-48ee-9198-3309b144ac29 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:15.168331 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268871c3-cb59-48ee-9198-3309b144ac29 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:15.173163 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdcf41-ae65-4081-b947-5934cff0ff93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:15.173764 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdcf41-ae65-4081-b947-5934cff0ff93 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:15.174041 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdcf41-ae65-4081-b947-5934cff0ff93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:15.174311 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdcf41-ae65-4081-b947-5934cff0ff93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:15.174977 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fdcf41-ae65-4081-b947-5934cff0ff93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:15.176630 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fdcf41-ae65-4081-b947-5934cff0ff93 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:15.187021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ec943-e284-41ed-986b-32844d440ee5 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:15.187385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1362/2732] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:06:15.191602 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdcf41-ae65-4081-b947-5934cff0ff93 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:15.194941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecb38b-59c4-4340-9a67-0e754cd6b751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:15.195574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecb38b-59c4-4340-9a67-0e754cd6b751 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:15.195840 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecb38b-59c4-4340-9a67-0e754cd6b751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:15.196112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecb38b-59c4-4340-9a67-0e754cd6b751 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:15.196748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecb38b-59c4-4340-9a67-0e754cd6b751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:15.198689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecb38b-59c4-4340-9a67-0e754cd6b751 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:15.201474 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdcf41-ae65-4081-b947-5934cff0ff93 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:15.206126 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7067e5-52c2-47cc-8e90-9048bf929c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:15.206788 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7067e5-52c2-47cc-8e90-9048bf929c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:15.207067 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7067e5-52c2-47cc-8e90-9048bf929c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:15.207392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7067e5-52c2-47cc-8e90-9048bf929c42 admin admin] PATH_INFO: `/v3/projects/0e1c8a957f314c19b527cc052fb1729d/users/fa0fd1a062134b658408687d3a8ee6fa/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:15.223040 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7067e5-52c2-47cc-8e90-9048bf929c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e1c8a957f314c19b527cc052fb1729d, user_id=fa0fd1a062134b658408687d3a8ee6fa, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:15.225674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7067e5-52c2-47cc-8e90-9048bf929c42 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:15.227780 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1371/2733] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:15 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:15.232248 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecb38b-59c4-4340-9a67-0e754cd6b751 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:15.239640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1363/2734] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:15 2022] PUT /identity/v3/projects/0e1c8a957f314c19b527cc052fb1729d/users/fa0fd1a062134b658408687d3a8ee6fa/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:06:15.243505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7067e5-52c2-47cc-8e90-9048bf929c42 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:15.295352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecb38b-59c4-4340-9a67-0e754cd6b751 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:15.308854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7067e5-52c2-47cc-8e90-9048bf929c42 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:15.318472 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d224a6-f0d5-4eb8-87ff-d23416f94ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:15.319130 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d224a6-f0d5-4eb8-87ff-d23416f94ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:15.319411 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d224a6-f0d5-4eb8-87ff-d23416f94ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:15.319686 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d224a6-f0d5-4eb8-87ff-d23416f94ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:15.320334 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d224a6-f0d5-4eb8-87ff-d23416f94ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:15.327943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d224a6-f0d5-4eb8-87ff-d23416f94ce6 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:15.342815 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1364/2735] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:15.344081 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d224a6-f0d5-4eb8-87ff-d23416f94ce6 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:15.354107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d224a6-f0d5-4eb8-87ff-d23416f94ce6 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:15.358049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6a868-61d5-4a29-b07d-6b80770dcd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:15.358606 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6a868-61d5-4a29-b07d-6b80770dcd51 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:15.358920 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6a868-61d5-4a29-b07d-6b80770dcd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:15.359199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6a868-61d5-4a29-b07d-6b80770dcd51 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:15.359446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423c5d8-a1b9-4bbf-bac2-b4e709a19963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:15.359836 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6a868-61d5-4a29-b07d-6b80770dcd51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:15.360083 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423c5d8-a1b9-4bbf-bac2-b4e709a19963 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:15.360361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423c5d8-a1b9-4bbf-bac2-b4e709a19963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:15.360627 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423c5d8-a1b9-4bbf-bac2-b4e709a19963 admin admin] PATH_INFO: `/v3/projects/0e1c8a957f314c19b527cc052fb1729d/users/fa0fd1a062134b658408687d3a8ee6fa/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:15.361746 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6a868-61d5-4a29-b07d-6b80770dcd51 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:15.362512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423c5d8-a1b9-4bbf-bac2-b4e709a19963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e1c8a957f314c19b527cc052fb1729d, user_id=fa0fd1a062134b658408687d3a8ee6fa, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:15.365138 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423c5d8-a1b9-4bbf-bac2-b4e709a19963 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:15.376156 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1365/2736] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:15 2022] PUT /identity/v3/projects/0e1c8a957f314c19b527cc052fb1729d/users/fa0fd1a062134b658408687d3a8ee6fa/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:06:15.387698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1372/2737] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:15 2022] POST /identity/v3/users => generated 295 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:15.387915 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323b822-07a1-472b-b9d6-1990072bc7a8 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:15.387915 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323b822-07a1-472b-b9d6-1990072bc7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:15.387915 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323b822-07a1-472b-b9d6-1990072bc7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:15.388842 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6a868-61d5-4a29-b07d-6b80770dcd51 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:15.399323 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6a868-61d5-4a29-b07d-6b80770dcd51 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:15.418136 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-4323b822-07a1-472b-b9d6-1990072bc7a8 None None] MFA Rules not processed for user `fa0fd1a062134b658408687d3a8ee6fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:15.419400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-4323b822-07a1-472b-b9d6-1990072bc7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:15.433828 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:15.452833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:15.461426 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d893ca-aeaf-4205-bb26-fb965eaff6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:15.462052 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d893ca-aeaf-4205-bb26-fb965eaff6a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:15.462330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d893ca-aeaf-4205-bb26-fb965eaff6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:15.462620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d893ca-aeaf-4205-bb26-fb965eaff6a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:15.463765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d893ca-aeaf-4205-bb26-fb965eaff6a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:15.465390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d893ca-aeaf-4205-bb26-fb965eaff6a4 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:15.475638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1373/2738] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:06:15.480868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d893ca-aeaf-4205-bb26-fb965eaff6a4 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:15.491339 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d893ca-aeaf-4205-bb26-fb965eaff6a4 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:15.497674 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f79495-334d-4285-a41c-c113a5f85d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:15.499019 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f79495-334d-4285-a41c-c113a5f85d8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:15.499728 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1366/2739] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:06:15.499862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f79495-334d-4285-a41c-c113a5f85d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:15.500575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f79495-334d-4285-a41c-c113a5f85d8d admin admin] PATH_INFO: `/v3/projects/2ae85cd9d0724afdb34647fb11c094f2/users/b47e8e6bde284a5997a74b9a53f8b8f1/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:15.510801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323b822-07a1-472b-b9d6-1990072bc7a8 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:15.512371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-4323b822-07a1-472b-b9d6-1990072bc7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:15.517185 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f79495-334d-4285-a41c-c113a5f85d8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae85cd9d0724afdb34647fb11c094f2, user_id=b47e8e6bde284a5997a74b9a53f8b8f1, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:15.519891 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f79495-334d-4285-a41c-c113a5f85d8d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:15.533331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1374/2740] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:15 2022] PUT /identity/v3/projects/2ae85cd9d0724afdb34647fb11c094f2/users/b47e8e6bde284a5997a74b9a53f8b8f1/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:15.537180 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f79495-334d-4285-a41c-c113a5f85d8d admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:15.573744 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323b822-07a1-472b-b9d6-1990072bc7a8 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:15.609115 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f79495-334d-4285-a41c-c113a5f85d8d admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:15.613888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c048d-0147-41cd-84ff-7d874fc769da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:15.614496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c048d-0147-41cd-84ff-7d874fc769da admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:15.614809 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c048d-0147-41cd-84ff-7d874fc769da admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:15.615088 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c048d-0147-41cd-84ff-7d874fc769da admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:15.615726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c048d-0147-41cd-84ff-7d874fc769da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:15.617336 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c048d-0147-41cd-84ff-7d874fc769da admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:15.626596 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1561a3-457f-4c98-8c22-a519bd7e3d6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:15.627243 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1561a3-457f-4c98-8c22-a519bd7e3d6d service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:15.627752 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1561a3-457f-4c98-8c22-a519bd7e3d6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:15.627886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1375/2741] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:06:15.627974 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1561a3-457f-4c98-8c22-a519bd7e3d6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:15.631636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c048d-0147-41cd-84ff-7d874fc769da admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:15.641253 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c048d-0147-41cd-84ff-7d874fc769da admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:15.645978 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2d12b-492d-49fc-9dce-1abd4f4f3f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:15.646620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2d12b-492d-49fc-9dce-1abd4f4f3f05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:15.646933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2d12b-492d-49fc-9dce-1abd4f4f3f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:15.647201 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2d12b-492d-49fc-9dce-1abd4f4f3f05 admin admin] PATH_INFO: `/v3/projects/2ae85cd9d0724afdb34647fb11c094f2/users/b47e8e6bde284a5997a74b9a53f8b8f1/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:15.649126 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2d12b-492d-49fc-9dce-1abd4f4f3f05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae85cd9d0724afdb34647fb11c094f2, user_id=b47e8e6bde284a5997a74b9a53f8b8f1, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:15.652485 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2d12b-492d-49fc-9dce-1abd4f4f3f05 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:15.664273 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1376/2742] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:15 2022] PUT /identity/v3/projects/2ae85cd9d0724afdb34647fb11c094f2/users/b47e8e6bde284a5997a74b9a53f8b8f1/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:06:15.675174 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0efc0-3288-4809-860f-c457a39910b3 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:15.675461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0efc0-3288-4809-860f-c457a39910b3 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:15.675741 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0efc0-3288-4809-860f-c457a39910b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:15.693818 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1561a3-457f-4c98-8c22-a519bd7e3d6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:15.695332 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1561a3-457f-4c98-8c22-a519bd7e3d6d service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:15.704856 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-6da0efc0-3288-4809-860f-c457a39910b3 None None] MFA Rules not processed for user `b47e8e6bde284a5997a74b9a53f8b8f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:15.705021 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1367/2743] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:06:15.706503 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-6da0efc0-3288-4809-860f-c457a39910b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:15.788894 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1377/2744] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:06:15.801516 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1561a3-457f-4c98-8c22-a519bd7e3d6d service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:15.811808 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1561a3-457f-4c98-8c22-a519bd7e3d6d service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:15.861714 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:15.883292 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb20f7-33c2-4923-bead-278b1a5d1a05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:15.883905 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb20f7-33c2-4923-bead-278b1a5d1a05 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:15.884186 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb20f7-33c2-4923-bead-278b1a5d1a05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:15.884458 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb20f7-33c2-4923-bead-278b1a5d1a05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:15.971430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb20f7-33c2-4923-bead-278b1a5d1a05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:15.973145 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb20f7-33c2-4923-bead-278b1a5d1a05 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:15.982753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1368/2745] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:16.311153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0efc0-3288-4809-860f-c457a39910b3 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:16.349291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:16.397940 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0efc0-3288-4809-860f-c457a39910b3 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:16.403712 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ef582-233d-4736-8ae4-2149040dd922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:16.404051 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ef582-233d-4736-8ae4-2149040dd922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:16.404331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ef582-233d-4736-8ae4-2149040dd922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:16.404603 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ef582-233d-4736-8ae4-2149040dd922 admin admin] PATH_INFO: `/v3/users/5e7557cf9d3f4951846386b91197b7ee` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:16.405898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ef582-233d-4736-8ae4-2149040dd922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e7557cf9d3f4951846386b91197b7ee)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:16.407926 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ef582-233d-4736-8ae4-2149040dd922 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:16.463162 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-7d5ef582-233d-4736-8ae4-2149040dd922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e7557cf9d3f4951846386b91197b7ee'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:16.470865 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-7d5ef582-233d-4736-8ae4-2149040dd922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e7557cf9d3f4951846386b91197b7ee'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:16.478147 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-7d5ef582-233d-4736-8ae4-2149040dd922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e7557cf9d3f4951846386b91197b7ee'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:16.490269 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-7d5ef582-233d-4736-8ae4-2149040dd922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e7557cf9d3f4951846386b91197b7ee'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:16.490960 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-7d5ef582-233d-4736-8ae4-2149040dd922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e7557cf9d3f4951846386b91197b7ee'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:16.497427 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1378/2746] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:16 2022] DELETE /identity/v3/users/5e7557cf9d3f4951846386b91197b7ee => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:16.502214 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad729dc-4e05-433c-9397-b856736e5786 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:16.502570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad729dc-4e05-433c-9397-b856736e5786 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:16.502966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad729dc-4e05-433c-9397-b856736e5786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:16.522538 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-7ad729dc-4e05-433c-9397-b856736e5786 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:16.523972 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7ad729dc-4e05-433c-9397-b856736e5786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:16.590014 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1369/2747] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:06:16.599868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ef582-233d-4736-8ae4-2149040dd922 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:16.601283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7d5ef582-233d-4736-8ae4-2149040dd922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:16.659533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ef582-233d-4736-8ae4-2149040dd922 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:16.663150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b219e0-8f45-4bbd-9b79-748c9f703254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:16.663749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b219e0-8f45-4bbd-9b79-748c9f703254 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:16.664023 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b219e0-8f45-4bbd-9b79-748c9f703254 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:16.664286 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b219e0-8f45-4bbd-9b79-748c9f703254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:16.673833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b219e0-8f45-4bbd-9b79-748c9f703254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:16.675291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b219e0-8f45-4bbd-9b79-748c9f703254 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:16.685259 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1379/2748] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:16.735398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad729dc-4e05-433c-9397-b856736e5786 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:16.736919 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7ad729dc-4e05-433c-9397-b856736e5786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:16.831192 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad729dc-4e05-433c-9397-b856736e5786 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:16.836755 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953adc4-bf07-4b5f-ac98-99d185c45bf2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b6b34b4ee5492fb1f0699716e5118b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:16.837360 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953adc4-bf07-4b5f-ac98-99d185c45bf2 service placement] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:16.837640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953adc4-bf07-4b5f-ac98-99d185c45bf2 service placement] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:16.837913 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953adc4-bf07-4b5f-ac98-99d185c45bf2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:16.839701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-1953adc4-bf07-4b5f-ac98-99d185c45bf2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:16.853160 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953adc4-bf07-4b5f-ac98-99d185c45bf2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:16.855799 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953adc4-bf07-4b5f-ac98-99d185c45bf2 service placement] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:16.860527 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:16.868193 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1370/2749] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:16.972905 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b219e0-8f45-4bbd-9b79-748c9f703254 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:16.974412 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b4b219e0-8f45-4bbd-9b79-748c9f703254 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:16.989590 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b219e0-8f45-4bbd-9b79-748c9f703254 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:16.995376 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914b674-8664-4160-ac63-0101c4f9ca71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:16.996026 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914b674-8664-4160-ac63-0101c4f9ca71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:16.996309 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914b674-8664-4160-ac63-0101c4f9ca71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:16.998382 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914b674-8664-4160-ac63-0101c4f9ca71 admin admin] PATH_INFO: `/v3/projects/f3c4548dcad04f1b971ed0c24ec3122e` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:16.999669 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914b674-8664-4160-ac63-0101c4f9ca71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3c4548dcad04f1b971ed0c24ec3122e)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:17.001676 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914b674-8664-4160-ac63-0101c4f9ca71 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:17.048790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:17.060539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-5914b674-8664-4160-ac63-0101c4f9ca71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3c4548dcad04f1b971ed0c24ec3122e', 'request_id': 'req-5914b674-8664-4160-ac63-0101c4f9ca71'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:17.068335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-5914b674-8664-4160-ac63-0101c4f9ca71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3c4548dcad04f1b971ed0c24ec3122e', 'request_id': 'req-5914b674-8664-4160-ac63-0101c4f9ca71'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:17.074624 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-5914b674-8664-4160-ac63-0101c4f9ca71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3c4548dcad04f1b971ed0c24ec3122e', 'request_id': 'req-5914b674-8664-4160-ac63-0101c4f9ca71'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:17.094921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-5914b674-8664-4160-ac63-0101c4f9ca71 admin admin] The token cache is being invalidate because project f3c4548dcad04f1b971ed0c24ec3122e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:17.095179 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-5914b674-8664-4160-ac63-0101c4f9ca71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:17.095888 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-5914b674-8664-4160-ac63-0101c4f9ca71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:17.097401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1380/2750] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:16 2022] DELETE /identity/v3/projects/f3c4548dcad04f1b971ed0c24ec3122e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:17.102087 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953adc4-bf07-4b5f-ac98-99d185c45bf2 service placement] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:17.114708 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953adc4-bf07-4b5f-ac98-99d185c45bf2 service placement] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:17.183092 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4ef27-ff02-4f17-bf33-42df60f8d381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:17.183749 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4ef27-ff02-4f17-bf33-42df60f8d381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:17.184031 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4ef27-ff02-4f17-bf33-42df60f8d381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:17.184312 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4ef27-ff02-4f17-bf33-42df60f8d381 admin admin] PATH_INFO: `/v3/users/fa0fd1a062134b658408687d3a8ee6fa` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:17.185475 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4ef27-ff02-4f17-bf33-42df60f8d381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa0fd1a062134b658408687d3a8ee6fa)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:17.187517 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4ef27-ff02-4f17-bf33-42df60f8d381 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:17.237558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e4f4ef27-ff02-4f17-bf33-42df60f8d381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa0fd1a062134b658408687d3a8ee6fa'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:17.247222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e4f4ef27-ff02-4f17-bf33-42df60f8d381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa0fd1a062134b658408687d3a8ee6fa'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:17.247854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e4f4ef27-ff02-4f17-bf33-42df60f8d381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa0fd1a062134b658408687d3a8ee6fa'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:17.262980 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e4f4ef27-ff02-4f17-bf33-42df60f8d381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa0fd1a062134b658408687d3a8ee6fa'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:17.263740 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e4f4ef27-ff02-4f17-bf33-42df60f8d381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa0fd1a062134b658408687d3a8ee6fa'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:17.278619 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1371/2751] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:17 2022] DELETE /identity/v3/users/fa0fd1a062134b658408687d3a8ee6fa => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:17.381980 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914b674-8664-4160-ac63-0101c4f9ca71 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:17.383674 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-5914b674-8664-4160-ac63-0101c4f9ca71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:17.467368 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914b674-8664-4160-ac63-0101c4f9ca71 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:17.531639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a670894-fe39-4e05-8293-9c04c486e310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:17.532624 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a670894-fe39-4e05-8293-9c04c486e310 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:17.532901 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a670894-fe39-4e05-8293-9c04c486e310 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:17.533118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a670894-fe39-4e05-8293-9c04c486e310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:17.535039 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7a670894-fe39-4e05-8293-9c04c486e310 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:17.602653 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4ef27-ff02-4f17-bf33-42df60f8d381 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:17.604990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e4f4ef27-ff02-4f17-bf33-42df60f8d381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:17.632476 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a670894-fe39-4e05-8293-9c04c486e310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:17.634205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a670894-fe39-4e05-8293-9c04c486e310 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:17.652194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1381/2752] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:17.706073 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4ef27-ff02-4f17-bf33-42df60f8d381 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:17.712884 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f580f94-1b21-45f5-a6f5-ebdce757e395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:17.714203 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f580f94-1b21-45f5-a6f5-ebdce757e395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:17.714426 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f580f94-1b21-45f5-a6f5-ebdce757e395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:17.714694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f580f94-1b21-45f5-a6f5-ebdce757e395 admin admin] PATH_INFO: `/v3/projects/0e1c8a957f314c19b527cc052fb1729d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:17.715956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f580f94-1b21-45f5-a6f5-ebdce757e395 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e1c8a957f314c19b527cc052fb1729d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:17.717945 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f580f94-1b21-45f5-a6f5-ebdce757e395 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:17.743764 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:17.771971 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0f580f94-1b21-45f5-a6f5-ebdce757e395 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e1c8a957f314c19b527cc052fb1729d', 'request_id': 'req-0f580f94-1b21-45f5-a6f5-ebdce757e395'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:17.780400 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0f580f94-1b21-45f5-a6f5-ebdce757e395 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e1c8a957f314c19b527cc052fb1729d', 'request_id': 'req-0f580f94-1b21-45f5-a6f5-ebdce757e395'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:17.787339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0f580f94-1b21-45f5-a6f5-ebdce757e395 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e1c8a957f314c19b527cc052fb1729d', 'request_id': 'req-0f580f94-1b21-45f5-a6f5-ebdce757e395'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:17.797910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0f580f94-1b21-45f5-a6f5-ebdce757e395 admin admin] The token cache is being invalidate because project 0e1c8a957f314c19b527cc052fb1729d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:17.798205 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0f580f94-1b21-45f5-a6f5-ebdce757e395 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:17.798887 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0f580f94-1b21-45f5-a6f5-ebdce757e395 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:17.800531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1372/2753] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:17 2022] DELETE /identity/v3/projects/0e1c8a957f314c19b527cc052fb1729d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:17.805506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f74f69-4f56-4695-ad1b-56ba462be72c None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:17.805791 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f74f69-4f56-4695-ad1b-56ba462be72c None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:17.806064 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f74f69-4f56-4695-ad1b-56ba462be72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:17.827573 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-b3f74f69-4f56-4695-ad1b-56ba462be72c None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:17.828799 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b3f74f69-4f56-4695-ad1b-56ba462be72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:17.854899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:17.895170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1382/2754] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:17.903182 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c2a26-2b71-4a1c-b64c-73fe80ff9100 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:17.903548 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c2a26-2b71-4a1c-b64c-73fe80ff9100 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:17.903835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c2a26-2b71-4a1c-b64c-73fe80ff9100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:17.925043 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-949c2a26-2b71-4a1c-b64c-73fe80ff9100 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:17.926198 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-949c2a26-2b71-4a1c-b64c-73fe80ff9100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:17.935286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1373/2755] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:17.939519 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f74f69-4f56-4695-ad1b-56ba462be72c None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:17.951000 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f74f69-4f56-4695-ad1b-56ba462be72c None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:17.956136 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca678b-b93c-4159-9fe3-488af471ed54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:17.956750 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca678b-b93c-4159-9fe3-488af471ed54 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:17.957156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca678b-b93c-4159-9fe3-488af471ed54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:17.957441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca678b-b93c-4159-9fe3-488af471ed54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:17.958089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca678b-b93c-4159-9fe3-488af471ed54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:06:17.958396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca678b-b93c-4159-9fe3-488af471ed54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:17.960306 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca678b-b93c-4159-9fe3-488af471ed54 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:17.972544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1383/2756] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:06:17.977168 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c2a26-2b71-4a1c-b64c-73fe80ff9100 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:17.986819 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c2a26-2b71-4a1c-b64c-73fe80ff9100 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:17.991812 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f6e71-fdef-4fc9-be0e-98b024500a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:17.992438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f6e71-fdef-4fc9-be0e-98b024500a62 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:17.992716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f6e71-fdef-4fc9-be0e-98b024500a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:17.992995 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f6e71-fdef-4fc9-be0e-98b024500a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:17.993623 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f6e71-fdef-4fc9-be0e-98b024500a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:17.996060 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f6e71-fdef-4fc9-be0e-98b024500a62 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:18.028614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1374/2757] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:17 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:18.033154 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca678b-b93c-4159-9fe3-488af471ed54 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:18.045410 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca678b-b93c-4159-9fe3-488af471ed54 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:18.112036 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5e73c-7313-4685-86f4-ccb3ce5791c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:18.112789 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5e73c-7313-4685-86f4-ccb3ce5791c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:18.113075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5e73c-7313-4685-86f4-ccb3ce5791c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:18.113351 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5e73c-7313-4685-86f4-ccb3ce5791c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:18.114004 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5e73c-7313-4685-86f4-ccb3ce5791c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:18.115971 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5e73c-7313-4685-86f4-ccb3ce5791c2 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:18.141241 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1384/2758] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:18 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:18.146870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f6e71-fdef-4fc9-be0e-98b024500a62 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:18.158152 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f6e71-fdef-4fc9-be0e-98b024500a62 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:18.162972 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18abf0-e263-4e6a-aa32-4ac626e8ba89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:18.163576 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18abf0-e263-4e6a-aa32-4ac626e8ba89 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:18.163855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18abf0-e263-4e6a-aa32-4ac626e8ba89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:18.164132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b18abf0-e263-4e6a-aa32-4ac626e8ba89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:18.164773 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18abf0-e263-4e6a-aa32-4ac626e8ba89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:18.166361 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b18abf0-e263-4e6a-aa32-4ac626e8ba89 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:18.177138 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1375/2759] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:18.181639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5e73c-7313-4685-86f4-ccb3ce5791c2 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:18.193106 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5e73c-7313-4685-86f4-ccb3ce5791c2 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:18.197864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef0634-8a14-4437-ad06-d64d1ddd573d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:18.198620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef0634-8a14-4437-ad06-d64d1ddd573d admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:18.199015 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef0634-8a14-4437-ad06-d64d1ddd573d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:18.199292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef0634-8a14-4437-ad06-d64d1ddd573d admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:18.200007 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef0634-8a14-4437-ad06-d64d1ddd573d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:18.201620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef0634-8a14-4437-ad06-d64d1ddd573d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:18.213276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1385/2760] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:06:18.217533 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18abf0-e263-4e6a-aa32-4ac626e8ba89 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:18.227689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b18abf0-e263-4e6a-aa32-4ac626e8ba89 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:18.232834 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6e0c2-e37e-40a9-ae95-92deba653f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:18.233489 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6e0c2-e37e-40a9-ae95-92deba653f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:18.233768 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6e0c2-e37e-40a9-ae95-92deba653f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:18.234039 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6e0c2-e37e-40a9-ae95-92deba653f9a admin admin] PATH_INFO: `/v3/projects/f76e148221c049f5823cca41b4665e54/users/ed638264c0ed4d2aace25b80c7e85d62/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:18.249818 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6e0c2-e37e-40a9-ae95-92deba653f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f76e148221c049f5823cca41b4665e54, user_id=ed638264c0ed4d2aace25b80c7e85d62, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:18.252699 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6e0c2-e37e-40a9-ae95-92deba653f9a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:18.266238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1376/2761] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:18 2022] PUT /identity/v3/projects/f76e148221c049f5823cca41b4665e54/users/ed638264c0ed4d2aace25b80c7e85d62/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:18.336994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe292b-33a7-4582-a432-f8f856d3ae18 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:18.337280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe292b-33a7-4582-a432-f8f856d3ae18 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:18.337555 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe292b-33a7-4582-a432-f8f856d3ae18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:18.367115 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-3ffe292b-33a7-4582-a432-f8f856d3ae18 None None] MFA Rules not processed for user `ed638264c0ed4d2aace25b80c7e85d62`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:18.368397 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-3ffe292b-33a7-4582-a432-f8f856d3ae18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:18.435128 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1386/2762] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:18.439715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6e0c2-e37e-40a9-ae95-92deba653f9a admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:18.506509 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6e0c2-e37e-40a9-ae95-92deba653f9a admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:18.511589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710812d-e35c-4976-b2e7-7184837b05ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:18.512206 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710812d-e35c-4976-b2e7-7184837b05ce admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:18.512487 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710812d-e35c-4976-b2e7-7184837b05ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:18.512759 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710812d-e35c-4976-b2e7-7184837b05ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:18.513413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710812d-e35c-4976-b2e7-7184837b05ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:18.515395 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710812d-e35c-4976-b2e7-7184837b05ce admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:18.543325 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1377/2763] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:18 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:18.547597 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe292b-33a7-4582-a432-f8f856d3ae18 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:18.559885 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe292b-33a7-4582-a432-f8f856d3ae18 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:18.580582 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:18.620430 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb00613-86b7-4117-83b0-36f24fceea8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:18.621054 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00613-86b7-4117-83b0-36f24fceea8d admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:18.621332 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00613-86b7-4117-83b0-36f24fceea8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:18.621605 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00613-86b7-4117-83b0-36f24fceea8d admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:18.623250 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb00613-86b7-4117-83b0-36f24fceea8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:18.624243 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb00613-86b7-4117-83b0-36f24fceea8d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:18.645602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1387/2764] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:18 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:18.649631 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710812d-e35c-4976-b2e7-7184837b05ce admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:18.661484 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710812d-e35c-4976-b2e7-7184837b05ce admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:18.666417 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a944a6f-a635-402d-8249-c881c658811e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:18.667600 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a944a6f-a635-402d-8249-c881c658811e admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:18.667879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a944a6f-a635-402d-8249-c881c658811e admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:18.668150 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a944a6f-a635-402d-8249-c881c658811e admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:18.668791 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a944a6f-a635-402d-8249-c881c658811e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:18.670392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a944a6f-a635-402d-8249-c881c658811e admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:18.681689 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1378/2765] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:18.685638 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb00613-86b7-4117-83b0-36f24fceea8d admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:18.695854 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb00613-86b7-4117-83b0-36f24fceea8d admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:18.701508 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091dc851-d483-4575-a47e-105b33c57954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:18.701775 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dc851-d483-4575-a47e-105b33c57954 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:18.702053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dc851-d483-4575-a47e-105b33c57954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:18.702460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dc851-d483-4575-a47e-105b33c57954 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:18.703139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dc851-d483-4575-a47e-105b33c57954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:18.704733 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dc851-d483-4575-a47e-105b33c57954 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:18.716477 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1388/2766] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:18.720559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a944a6f-a635-402d-8249-c881c658811e admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:18.734183 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a944a6f-a635-402d-8249-c881c658811e admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:18.739319 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0363243-df3e-42cd-8626-b999adc1f110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:18.739990 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0363243-df3e-42cd-8626-b999adc1f110 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:18.740372 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0363243-df3e-42cd-8626-b999adc1f110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:18.740645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0363243-df3e-42cd-8626-b999adc1f110 admin admin] PATH_INFO: `/v3/projects/90a6011e99764af6863c0a9bcf3c8c06/users/8138da84ffff4d28ab3a40faf8061ac6/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:18.757447 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0363243-df3e-42cd-8626-b999adc1f110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a6011e99764af6863c0a9bcf3c8c06, user_id=8138da84ffff4d28ab3a40faf8061ac6, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:18.760517 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0363243-df3e-42cd-8626-b999adc1f110 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:18.762883 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:18.776902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1379/2767] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:18 2022] PUT /identity/v3/projects/90a6011e99764af6863c0a9bcf3c8c06/users/8138da84ffff4d28ab3a40faf8061ac6/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:18.787232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9a906-9740-4413-8c58-7b356b976ff2 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:18.787537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9a906-9740-4413-8c58-7b356b976ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:18.787834 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9a906-9740-4413-8c58-7b356b976ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:18.817983 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-55b9a906-9740-4413-8c58-7b356b976ff2 None None] MFA Rules not processed for user `8138da84ffff4d28ab3a40faf8061ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:18.819327 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-55b9a906-9740-4413-8c58-7b356b976ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:18.887698 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1389/2768] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:18.892291 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0363243-df3e-42cd-8626-b999adc1f110 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:18.957848 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0363243-df3e-42cd-8626-b999adc1f110 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:18.962703 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da80f7-f346-4228-8c44-c8564d4dd29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:18.963337 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da80f7-f346-4228-8c44-c8564d4dd29a admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:18.964174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da80f7-f346-4228-8c44-c8564d4dd29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:18.964450 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da80f7-f346-4228-8c44-c8564d4dd29a admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:18.965085 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51da80f7-f346-4228-8c44-c8564d4dd29a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:18.967016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51da80f7-f346-4228-8c44-c8564d4dd29a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:18.995119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1380/2769] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:18 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:18.998913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9a906-9740-4413-8c58-7b356b976ff2 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:19.010131 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9a906-9740-4413-8c58-7b356b976ff2 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:19.068752 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a89a6a-0490-45b8-9a0e-12f6f82c1b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:19.069378 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a89a6a-0490-45b8-9a0e-12f6f82c1b60 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:19.069655 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a89a6a-0490-45b8-9a0e-12f6f82c1b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:19.069929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a89a6a-0490-45b8-9a0e-12f6f82c1b60 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:19.070569 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a89a6a-0490-45b8-9a0e-12f6f82c1b60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:19.072517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a89a6a-0490-45b8-9a0e-12f6f82c1b60 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:19.097110 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1390/2770] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:18 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:19.100902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da80f7-f346-4228-8c44-c8564d4dd29a admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:19.112233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da80f7-f346-4228-8c44-c8564d4dd29a admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:19.117088 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae62a1d-9c09-44f8-bc25-b84891b09437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:19.117715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae62a1d-9c09-44f8-bc25-b84891b09437 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:19.118087 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae62a1d-9c09-44f8-bc25-b84891b09437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:19.118282 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae62a1d-9c09-44f8-bc25-b84891b09437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:19.119011 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae62a1d-9c09-44f8-bc25-b84891b09437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:19.120676 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae62a1d-9c09-44f8-bc25-b84891b09437 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:19.132293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1381/2771] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:19.137240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a89a6a-0490-45b8-9a0e-12f6f82c1b60 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:19.148089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a89a6a-0490-45b8-9a0e-12f6f82c1b60 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:19.153936 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa84a9-31dd-4686-ab8a-57f8622465ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:19.154635 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa84a9-31dd-4686-ab8a-57f8622465ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:19.154954 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa84a9-31dd-4686-ab8a-57f8622465ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:19.155233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa84a9-31dd-4686-ab8a-57f8622465ff admin admin] PATH_INFO: `/v3/projects/34310f5d39ca41e2b7c6fa200ba63dfd/users/cad07889d95a4e6f9e01b6ec328ddd80/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:19.171909 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa84a9-31dd-4686-ab8a-57f8622465ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=34310f5d39ca41e2b7c6fa200ba63dfd, user_id=cad07889d95a4e6f9e01b6ec328ddd80, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:19.174627 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa84a9-31dd-4686-ab8a-57f8622465ff admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:19.183922 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:19.190977 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1391/2772] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:19 2022] PUT /identity/v3/projects/34310f5d39ca41e2b7c6fa200ba63dfd/users/cad07889d95a4e6f9e01b6ec328ddd80/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:19.194813 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae62a1d-9c09-44f8-bc25-b84891b09437 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:19.206008 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae62a1d-9c09-44f8-bc25-b84891b09437 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:19.267133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207cc03a-86a5-4b11-83f9-475be05d3b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:19.267660 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cc03a-86a5-4b11-83f9-475be05d3b40 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:19.267943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cc03a-86a5-4b11-83f9-475be05d3b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:19.268239 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207cc03a-86a5-4b11-83f9-475be05d3b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:19.268900 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cc03a-86a5-4b11-83f9-475be05d3b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:19.270518 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207cc03a-86a5-4b11-83f9-475be05d3b40 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:19.285028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1382/2773] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:19.286495 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa84a9-31dd-4686-ab8a-57f8622465ff admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:19.297088 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa84a9-31dd-4686-ab8a-57f8622465ff admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:19.301831 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9cb6d-a11a-43ce-91db-0cf241a35bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:19.302496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9cb6d-a11a-43ce-91db-0cf241a35bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:19.302803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9cb6d-a11a-43ce-91db-0cf241a35bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:19.303082 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9cb6d-a11a-43ce-91db-0cf241a35bd5 admin admin] PATH_INFO: `/v3/projects/34310f5d39ca41e2b7c6fa200ba63dfd/users/cad07889d95a4e6f9e01b6ec328ddd80/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:19.310862 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9cb6d-a11a-43ce-91db-0cf241a35bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34310f5d39ca41e2b7c6fa200ba63dfd, user_id=cad07889d95a4e6f9e01b6ec328ddd80, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:19.313524 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9cb6d-a11a-43ce-91db-0cf241a35bd5 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:19.325427 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1392/2774] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:19 2022] PUT /identity/v3/projects/34310f5d39ca41e2b7c6fa200ba63dfd/users/cad07889d95a4e6f9e01b6ec328ddd80/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:19.337367 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a40a18-13a5-4b8e-be53-7da10e227835 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:19.338311 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a40a18-13a5-4b8e-be53-7da10e227835 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:19.338311 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a40a18-13a5-4b8e-be53-7da10e227835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:19.372966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-68a40a18-13a5-4b8e-be53-7da10e227835 None None] MFA Rules not processed for user `cad07889d95a4e6f9e01b6ec328ddd80`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:19.374210 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-68a40a18-13a5-4b8e-be53-7da10e227835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:19.429278 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:19.451541 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1383/2775] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:19.463382 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9cb6d-a11a-43ce-91db-0cf241a35bd5 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:19.465346 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-17d9cb6d-a11a-43ce-91db-0cf241a35bd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:19.546251 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9cb6d-a11a-43ce-91db-0cf241a35bd5 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:19.550330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c0100-55d9-4edc-aa02-33f8c9aa58f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:19.551661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c0100-55d9-4edc-aa02-33f8c9aa58f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:19.552002 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c0100-55d9-4edc-aa02-33f8c9aa58f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:19.552365 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c0100-55d9-4edc-aa02-33f8c9aa58f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:19.623729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c0100-55d9-4edc-aa02-33f8c9aa58f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:19.625314 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c0100-55d9-4edc-aa02-33f8c9aa58f8 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:19.635824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1393/2776] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:20.799054 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a40a18-13a5-4b8e-be53-7da10e227835 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:20.800494 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-68a40a18-13a5-4b8e-be53-7da10e227835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:20.813249 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a40a18-13a5-4b8e-be53-7da10e227835 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:20.817008 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e05d78-1963-4c25-ad57-3edf28e7f3ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:20.817613 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e05d78-1963-4c25-ad57-3edf28e7f3ed service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:20.817895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e05d78-1963-4c25-ad57-3edf28e7f3ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:20.818171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e05d78-1963-4c25-ad57-3edf28e7f3ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:20.819922 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-58e05d78-1963-4c25-ad57-3edf28e7f3ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:20.887609 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e05d78-1963-4c25-ad57-3edf28e7f3ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:20.889266 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e05d78-1963-4c25-ad57-3edf28e7f3ed service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:20.899181 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1384/2777] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:21.264788 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c0100-55d9-4edc-aa02-33f8c9aa58f8 service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:21.276446 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c0100-55d9-4edc-aa02-33f8c9aa58f8 service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:21.283170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cad8d1-af31-4700-8233-213f5ffa2e1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:21.284609 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cad8d1-af31-4700-8233-213f5ffa2e1a service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:21.284895 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cad8d1-af31-4700-8233-213f5ffa2e1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:21.285490 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cad8d1-af31-4700-8233-213f5ffa2e1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:21.399861 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cad8d1-af31-4700-8233-213f5ffa2e1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:21.402209 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cad8d1-af31-4700-8233-213f5ffa2e1a service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:21.415899 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1394/2778] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:22.684003 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e05d78-1963-4c25-ad57-3edf28e7f3ed service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:22.692720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e05d78-1963-4c25-ad57-3edf28e7f3ed service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:22.696391 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8fed1d-fc2a-4de3-9c50-cbb34b24baad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:22.696985 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fed1d-fc2a-4de3-9c50-cbb34b24baad service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:22.697272 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fed1d-fc2a-4de3-9c50-cbb34b24baad service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:22.697519 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fed1d-fc2a-4de3-9c50-cbb34b24baad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:22.706234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fed1d-fc2a-4de3-9c50-cbb34b24baad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:22.707688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8fed1d-fc2a-4de3-9c50-cbb34b24baad service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:22.717835 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1385/2779] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:27.660513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cad8d1-af31-4700-8233-213f5ffa2e1a service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:27.822466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:27.845304 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cad8d1-af31-4700-8233-213f5ffa2e1a service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:27.861241 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409b125-9c24-49e5-b45a-ae92c93b495a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:27.862084 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409b125-9c24-49e5-b45a-ae92c93b495a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:27.862363 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409b125-9c24-49e5-b45a-ae92c93b495a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:27.862633 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409b125-9c24-49e5-b45a-ae92c93b495a admin admin] PATH_INFO: `/v3/users/ed638264c0ed4d2aace25b80c7e85d62` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:27.863964 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9409b125-9c24-49e5-b45a-ae92c93b495a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed638264c0ed4d2aace25b80c7e85d62)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:27.865934 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9409b125-9c24-49e5-b45a-ae92c93b495a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:27.913089 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-9409b125-9c24-49e5-b45a-ae92c93b495a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed638264c0ed4d2aace25b80c7e85d62'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:27.924184 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-9409b125-9c24-49e5-b45a-ae92c93b495a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed638264c0ed4d2aace25b80c7e85d62'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:27.931279 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-9409b125-9c24-49e5-b45a-ae92c93b495a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed638264c0ed4d2aace25b80c7e85d62'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:27.942952 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-9409b125-9c24-49e5-b45a-ae92c93b495a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed638264c0ed4d2aace25b80c7e85d62'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:27.943583 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-9409b125-9c24-49e5-b45a-ae92c93b495a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed638264c0ed4d2aace25b80c7e85d62'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:27.949743 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1395/2780] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:27 2022] DELETE /identity/v3/users/ed638264c0ed4d2aace25b80c7e85d62 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:27.954329 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1277a6f-e6c6-4ac3-a821-e167e3f4e303 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:27.954832 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1277a6f-e6c6-4ac3-a821-e167e3f4e303 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:27.955184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1277a6f-e6c6-4ac3-a821-e167e3f4e303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:27.975097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-b1277a6f-e6c6-4ac3-a821-e167e3f4e303 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:27.976304 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b1277a6f-e6c6-4ac3-a821-e167e3f4e303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:28.035927 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1386/2781] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:06:28.045774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409b125-9c24-49e5-b45a-ae92c93b495a admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:28.047218 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-9409b125-9c24-49e5-b45a-ae92c93b495a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:28.128904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409b125-9c24-49e5-b45a-ae92c93b495a admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:28.132735 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f174f4-5b5c-4684-a174-3f8d2c4cf37b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:28.133222 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f174f4-5b5c-4684-a174-3f8d2c4cf37b service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:28.133504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f174f4-5b5c-4684-a174-3f8d2c4cf37b service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:28.133772 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f174f4-5b5c-4684-a174-3f8d2c4cf37b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:28.143232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f174f4-5b5c-4684-a174-3f8d2c4cf37b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:28.144617 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f174f4-5b5c-4684-a174-3f8d2c4cf37b service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:28.154624 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1396/2782] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:28.422322 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1277a6f-e6c6-4ac3-a821-e167e3f4e303 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:28.423715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-b1277a6f-e6c6-4ac3-a821-e167e3f4e303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:28.436111 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1277a6f-e6c6-4ac3-a821-e167e3f4e303 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:28.440749 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b348f-b49c-4fe7-8c1a-c1728e477409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:28.441391 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b348f-b49c-4fe7-8c1a-c1728e477409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:28.441672 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b348f-b49c-4fe7-8c1a-c1728e477409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:28.441947 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524b348f-b49c-4fe7-8c1a-c1728e477409 admin admin] PATH_INFO: `/v3/projects/f76e148221c049f5823cca41b4665e54` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:28.443132 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b348f-b49c-4fe7-8c1a-c1728e477409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f76e148221c049f5823cca41b4665e54)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:28.445049 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524b348f-b49c-4fe7-8c1a-c1728e477409 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:28.492275 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-524b348f-b49c-4fe7-8c1a-c1728e477409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f76e148221c049f5823cca41b4665e54', 'request_id': 'req-524b348f-b49c-4fe7-8c1a-c1728e477409'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:28.499845 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-524b348f-b49c-4fe7-8c1a-c1728e477409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f76e148221c049f5823cca41b4665e54', 'request_id': 'req-524b348f-b49c-4fe7-8c1a-c1728e477409'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:28.506230 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-524b348f-b49c-4fe7-8c1a-c1728e477409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f76e148221c049f5823cca41b4665e54', 'request_id': 'req-524b348f-b49c-4fe7-8c1a-c1728e477409'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:28.510822 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:28.519050 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-524b348f-b49c-4fe7-8c1a-c1728e477409 admin admin] The token cache is being invalidate because project f76e148221c049f5823cca41b4665e54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:28.519607 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-524b348f-b49c-4fe7-8c1a-c1728e477409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:28.520528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-524b348f-b49c-4fe7-8c1a-c1728e477409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:28.522047 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1387/2783] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:28 2022] DELETE /identity/v3/projects/f76e148221c049f5823cca41b4665e54 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:28.526139 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f174f4-5b5c-4684-a174-3f8d2c4cf37b service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:28.535942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f174f4-5b5c-4684-a174-3f8d2c4cf37b service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:28.573990 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:28.595525 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37804d-cdea-4ee9-b3bc-c8a33dcd93cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:28.596160 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37804d-cdea-4ee9-b3bc-c8a33dcd93cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:28.596439 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37804d-cdea-4ee9-b3bc-c8a33dcd93cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:28.596713 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37804d-cdea-4ee9-b3bc-c8a33dcd93cc admin admin] PATH_INFO: `/v3/users/8138da84ffff4d28ab3a40faf8061ac6` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:28.597840 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37804d-cdea-4ee9-b3bc-c8a33dcd93cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8138da84ffff4d28ab3a40faf8061ac6)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:28.599824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37804d-cdea-4ee9-b3bc-c8a33dcd93cc admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:28.649764 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6f37804d-cdea-4ee9-b3bc-c8a33dcd93cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8138da84ffff4d28ab3a40faf8061ac6'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:28.657825 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6f37804d-cdea-4ee9-b3bc-c8a33dcd93cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8138da84ffff4d28ab3a40faf8061ac6'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:28.665076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6f37804d-cdea-4ee9-b3bc-c8a33dcd93cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8138da84ffff4d28ab3a40faf8061ac6'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:28.677967 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6f37804d-cdea-4ee9-b3bc-c8a33dcd93cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8138da84ffff4d28ab3a40faf8061ac6'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:28.678642 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6f37804d-cdea-4ee9-b3bc-c8a33dcd93cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8138da84ffff4d28ab3a40faf8061ac6'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:28.685138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1397/2784] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:28 2022] DELETE /identity/v3/users/8138da84ffff4d28ab3a40faf8061ac6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:29.018244 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b348f-b49c-4fe7-8c1a-c1728e477409 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:29.019700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-524b348f-b49c-4fe7-8c1a-c1728e477409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:29.032942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524b348f-b49c-4fe7-8c1a-c1728e477409 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:29.090852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d25e3-139e-493a-9d2d-d3979e5452b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:29.091487 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d25e3-139e-493a-9d2d-d3979e5452b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:29.091762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d25e3-139e-493a-9d2d-d3979e5452b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:29.092031 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d25e3-139e-493a-9d2d-d3979e5452b1 admin admin] PATH_INFO: `/v3/projects/90a6011e99764af6863c0a9bcf3c8c06` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:29.093161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d25e3-139e-493a-9d2d-d3979e5452b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90a6011e99764af6863c0a9bcf3c8c06)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:29.095084 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d25e3-139e-493a-9d2d-d3979e5452b1 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:29.140314 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e33d25e3-139e-493a-9d2d-d3979e5452b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90a6011e99764af6863c0a9bcf3c8c06', 'request_id': 'req-e33d25e3-139e-493a-9d2d-d3979e5452b1'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:29.147309 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e33d25e3-139e-493a-9d2d-d3979e5452b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90a6011e99764af6863c0a9bcf3c8c06', 'request_id': 'req-e33d25e3-139e-493a-9d2d-d3979e5452b1'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:29.153950 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e33d25e3-139e-493a-9d2d-d3979e5452b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90a6011e99764af6863c0a9bcf3c8c06', 'request_id': 'req-e33d25e3-139e-493a-9d2d-d3979e5452b1'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:29.164524 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e33d25e3-139e-493a-9d2d-d3979e5452b1 admin admin] The token cache is being invalidate because project 90a6011e99764af6863c0a9bcf3c8c06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:29.164811 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e33d25e3-139e-493a-9d2d-d3979e5452b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:29.165438 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e33d25e3-139e-493a-9d2d-d3979e5452b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:29.167480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1388/2785] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:29 2022] DELETE /identity/v3/projects/90a6011e99764af6863c0a9bcf3c8c06 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:29.171629 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37804d-cdea-4ee9-b3bc-c8a33dcd93cc admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:29.173081 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-6f37804d-cdea-4ee9-b3bc-c8a33dcd93cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:29.185808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37804d-cdea-4ee9-b3bc-c8a33dcd93cc admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:29.240815 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51216385-88b2-4cfd-9b5b-adc36b9df172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:29.241366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51216385-88b2-4cfd-9b5b-adc36b9df172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:29.241644 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51216385-88b2-4cfd-9b5b-adc36b9df172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:29.241912 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51216385-88b2-4cfd-9b5b-adc36b9df172 admin admin] PATH_INFO: `/v3/users/cad07889d95a4e6f9e01b6ec328ddd80` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:29.243049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51216385-88b2-4cfd-9b5b-adc36b9df172 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cad07889d95a4e6f9e01b6ec328ddd80)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:29.244989 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51216385-88b2-4cfd-9b5b-adc36b9df172 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:29.290258 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-51216385-88b2-4cfd-9b5b-adc36b9df172 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cad07889d95a4e6f9e01b6ec328ddd80'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:29.299268 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-51216385-88b2-4cfd-9b5b-adc36b9df172 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cad07889d95a4e6f9e01b6ec328ddd80'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:29.304615 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:29.307121 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-51216385-88b2-4cfd-9b5b-adc36b9df172 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cad07889d95a4e6f9e01b6ec328ddd80'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:29.320797 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-51216385-88b2-4cfd-9b5b-adc36b9df172 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cad07889d95a4e6f9e01b6ec328ddd80'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:29.321500 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-51216385-88b2-4cfd-9b5b-adc36b9df172 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cad07889d95a4e6f9e01b6ec328ddd80'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:29.327706 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1398/2786] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:29 2022] DELETE /identity/v3/users/cad07889d95a4e6f9e01b6ec328ddd80 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:29.601160 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d25e3-139e-493a-9d2d-d3979e5452b1 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:29.602605 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-e33d25e3-139e-493a-9d2d-d3979e5452b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:29.615670 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d25e3-139e-493a-9d2d-d3979e5452b1 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:29.674928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f523be-52ed-4dae-aa5d-797d3d1608b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:29.675569 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f523be-52ed-4dae-aa5d-797d3d1608b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:29.675848 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f523be-52ed-4dae-aa5d-797d3d1608b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:29.676120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f523be-52ed-4dae-aa5d-797d3d1608b7 admin admin] PATH_INFO: `/v3/projects/34310f5d39ca41e2b7c6fa200ba63dfd` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:29.677233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f523be-52ed-4dae-aa5d-797d3d1608b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34310f5d39ca41e2b7c6fa200ba63dfd)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:29.679172 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f523be-52ed-4dae-aa5d-797d3d1608b7 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:29.714495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:29.733246 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f2f523be-52ed-4dae-aa5d-797d3d1608b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34310f5d39ca41e2b7c6fa200ba63dfd', 'request_id': 'req-f2f523be-52ed-4dae-aa5d-797d3d1608b7'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:29.740989 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f2f523be-52ed-4dae-aa5d-797d3d1608b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34310f5d39ca41e2b7c6fa200ba63dfd', 'request_id': 'req-f2f523be-52ed-4dae-aa5d-797d3d1608b7'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:29.748248 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f2f523be-52ed-4dae-aa5d-797d3d1608b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34310f5d39ca41e2b7c6fa200ba63dfd', 'request_id': 'req-f2f523be-52ed-4dae-aa5d-797d3d1608b7'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:29.759308 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f2f523be-52ed-4dae-aa5d-797d3d1608b7 admin admin] The token cache is being invalidate because project 34310f5d39ca41e2b7c6fa200ba63dfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:29.759599 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f2f523be-52ed-4dae-aa5d-797d3d1608b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:29.760347 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f2f523be-52ed-4dae-aa5d-797d3d1608b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:29.762117 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1389/2787] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:29 2022] DELETE /identity/v3/projects/34310f5d39ca41e2b7c6fa200ba63dfd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:29.767192 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e77d9-5f1c-44a8-9081-feb524103259 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:29.767705 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e77d9-5f1c-44a8-9081-feb524103259 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:29.767819 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e77d9-5f1c-44a8-9081-feb524103259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:29.788490 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-b54e77d9-5f1c-44a8-9081-feb524103259 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:29.789628 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b54e77d9-5f1c-44a8-9081-feb524103259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:29.850423 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1399/2788] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:29.859106 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340783e2-5670-49fe-a6ff-7f987bb1eeb0 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:29.859374 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340783e2-5670-49fe-a6ff-7f987bb1eeb0 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:29.859642 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340783e2-5670-49fe-a6ff-7f987bb1eeb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:29.880317 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-340783e2-5670-49fe-a6ff-7f987bb1eeb0 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:29.881603 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-340783e2-5670-49fe-a6ff-7f987bb1eeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:29.890082 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1390/2789] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:29.894464 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e77d9-5f1c-44a8-9081-feb524103259 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:29.906155 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e77d9-5f1c-44a8-9081-feb524103259 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:29.913037 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28d115-b64a-404d-bae4-89392e2ecbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:29.913778 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28d115-b64a-404d-bae4-89392e2ecbdd admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:29.914123 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28d115-b64a-404d-bae4-89392e2ecbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:29.914462 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28d115-b64a-404d-bae4-89392e2ecbdd admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:29.915229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd28d115-b64a-404d-bae4-89392e2ecbdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:06:29.915942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd28d115-b64a-404d-bae4-89392e2ecbdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:29.918383 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd28d115-b64a-404d-bae4-89392e2ecbdd admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:29.936260 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1400/2790] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:06:29.937263 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340783e2-5670-49fe-a6ff-7f987bb1eeb0 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:29.948109 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340783e2-5670-49fe-a6ff-7f987bb1eeb0 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:29.953077 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18264eb-6a04-4831-af40-4268fac64383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:29.954133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18264eb-6a04-4831-af40-4268fac64383 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:29.954133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18264eb-6a04-4831-af40-4268fac64383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:29.954657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18264eb-6a04-4831-af40-4268fac64383 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:29.955712 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18264eb-6a04-4831-af40-4268fac64383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:29.958491 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18264eb-6a04-4831-af40-4268fac64383 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:30.012058 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28d115-b64a-404d-bae4-89392e2ecbdd admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:30.017122 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1391/2791] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:29 2022] POST /identity/v3/projects => generated 369 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:30.022190 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28d115-b64a-404d-bae4-89392e2ecbdd admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:30.091650 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734201d3-97ef-49fe-ab15-b590f5ef8c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:30.092525 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734201d3-97ef-49fe-ab15-b590f5ef8c15 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:30.092837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734201d3-97ef-49fe-ab15-b590f5ef8c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:30.093172 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734201d3-97ef-49fe-ab15-b590f5ef8c15 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:30.093740 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734201d3-97ef-49fe-ab15-b590f5ef8c15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:30.095665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734201d3-97ef-49fe-ab15-b590f5ef8c15 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:30.120248 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1401/2792] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:30 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:30.124420 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18264eb-6a04-4831-af40-4268fac64383 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:30.135596 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18264eb-6a04-4831-af40-4268fac64383 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:30.140274 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436eeb8c-1ed5-40cd-94f1-cc77edcbe425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:30.140877 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436eeb8c-1ed5-40cd-94f1-cc77edcbe425 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:30.141155 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436eeb8c-1ed5-40cd-94f1-cc77edcbe425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:30.141442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436eeb8c-1ed5-40cd-94f1-cc77edcbe425 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:30.142056 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436eeb8c-1ed5-40cd-94f1-cc77edcbe425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:30.143684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436eeb8c-1ed5-40cd-94f1-cc77edcbe425 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:30.153595 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1392/2793] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:30.157882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734201d3-97ef-49fe-ab15-b590f5ef8c15 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:30.168497 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734201d3-97ef-49fe-ab15-b590f5ef8c15 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:30.173183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881584f-3cfd-43c8-8bc8-1375052d5c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:30.173782 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881584f-3cfd-43c8-8bc8-1375052d5c93 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:30.174059 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881584f-3cfd-43c8-8bc8-1375052d5c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:30.174329 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881584f-3cfd-43c8-8bc8-1375052d5c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:30.175006 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881584f-3cfd-43c8-8bc8-1375052d5c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:30.176572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881584f-3cfd-43c8-8bc8-1375052d5c93 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:30.187168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1402/2794] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:30.191441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436eeb8c-1ed5-40cd-94f1-cc77edcbe425 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:30.202318 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436eeb8c-1ed5-40cd-94f1-cc77edcbe425 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:30.206917 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a38ce9-2c84-4a56-8c42-d848dc521d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:30.207597 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a38ce9-2c84-4a56-8c42-d848dc521d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:30.207875 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a38ce9-2c84-4a56-8c42-d848dc521d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:30.208148 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a38ce9-2c84-4a56-8c42-d848dc521d2b admin admin] PATH_INFO: `/v3/projects/6c7980f54e2645dfbb561a9d8313e234/users/0c8ee67696ec4441b29c6421c12e1276/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:30.223072 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a38ce9-2c84-4a56-8c42-d848dc521d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c7980f54e2645dfbb561a9d8313e234, user_id=0c8ee67696ec4441b29c6421c12e1276, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:30.225760 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a38ce9-2c84-4a56-8c42-d848dc521d2b admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:30.240549 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1393/2795] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:30 2022] PUT /identity/v3/projects/6c7980f54e2645dfbb561a9d8313e234/users/0c8ee67696ec4441b29c6421c12e1276/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:06:30.251740 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182992c5-72c0-45f2-8a6a-1adac32492d6 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:30.252082 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182992c5-72c0-45f2-8a6a-1adac32492d6 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:30.252367 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182992c5-72c0-45f2-8a6a-1adac32492d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:30.279681 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-182992c5-72c0-45f2-8a6a-1adac32492d6 None None] MFA Rules not processed for user `0c8ee67696ec4441b29c6421c12e1276`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:30.280877 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-182992c5-72c0-45f2-8a6a-1adac32492d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:30.313052 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:30.348133 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1403/2796] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:30.360237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a38ce9-2c84-4a56-8c42-d848dc521d2b admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:30.361939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-65a38ce9-2c84-4a56-8c42-d848dc521d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:30.421171 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a38ce9-2c84-4a56-8c42-d848dc521d2b admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:30.424950 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1969ba-fd92-4a7c-bf5b-18b3fbd6e35d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5248cb1c39412680cce0d9891fe1ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:30.425564 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1969ba-fd92-4a7c-bf5b-18b3fbd6e35d service glance] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:30.425845 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1969ba-fd92-4a7c-bf5b-18b3fbd6e35d service glance] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:30.426119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1969ba-fd92-4a7c-bf5b-18b3fbd6e35d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:30.435006 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1969ba-fd92-4a7c-bf5b-18b3fbd6e35d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:30.436435 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1969ba-fd92-4a7c-bf5b-18b3fbd6e35d service glance] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:30.445164 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1394/2797] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:40.345107 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=12 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:40.441234 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:44.885436 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182992c5-72c0-45f2-8a6a-1adac32492d6 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:44.887068 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-182992c5-72c0-45f2-8a6a-1adac32492d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:44.949671 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182992c5-72c0-45f2-8a6a-1adac32492d6 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:44.954156 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8522bf-a66f-462a-9299-b734632fb3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:44.954810 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8522bf-a66f-462a-9299-b734632fb3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:44.955093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8522bf-a66f-462a-9299-b734632fb3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:44.955366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8522bf-a66f-462a-9299-b734632fb3e6 admin admin] PATH_INFO: `/v3/users/5a3883f7400d4d51a3f4459ce3a79b56` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:44.956489 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8522bf-a66f-462a-9299-b734632fb3e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a3883f7400d4d51a3f4459ce3a79b56)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:44.958400 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8522bf-a66f-462a-9299-b734632fb3e6 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:45.005672 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-7b8522bf-a66f-462a-9299-b734632fb3e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a3883f7400d4d51a3f4459ce3a79b56'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:45.013064 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-7b8522bf-a66f-462a-9299-b734632fb3e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a3883f7400d4d51a3f4459ce3a79b56'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:45.020640 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-7b8522bf-a66f-462a-9299-b734632fb3e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a3883f7400d4d51a3f4459ce3a79b56'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:45.033302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-7b8522bf-a66f-462a-9299-b734632fb3e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a3883f7400d4d51a3f4459ce3a79b56'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:45.033961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-7b8522bf-a66f-462a-9299-b734632fb3e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a3883f7400d4d51a3f4459ce3a79b56'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:45.039788 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1404/2798] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:44 2022] DELETE /identity/v3/users/5a3883f7400d4d51a3f4459ce3a79b56 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:45.044401 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcc7ad-61e5-4ce9-aaf4-a4326d116135 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:45.044897 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcc7ad-61e5-4ce9-aaf4-a4326d116135 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:45.045073 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcc7ad-61e5-4ce9-aaf4-a4326d116135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:45.065406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-a4fcc7ad-61e5-4ce9-aaf4-a4326d116135 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:45.066654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a4fcc7ad-61e5-4ce9-aaf4-a4326d116135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:45.130855 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1395/2799] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:45.140979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8522bf-a66f-462a-9299-b734632fb3e6 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:45.142410 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7b8522bf-a66f-462a-9299-b734632fb3e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:45.198331 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8522bf-a66f-462a-9299-b734632fb3e6 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:45.202028 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6cc5c-3708-48ef-91dd-30cb0d52df23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:45.203173 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6cc5c-3708-48ef-91dd-30cb0d52df23 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:45.203453 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6cc5c-3708-48ef-91dd-30cb0d52df23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:45.203723 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6cc5c-3708-48ef-91dd-30cb0d52df23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:45.213183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6cc5c-3708-48ef-91dd-30cb0d52df23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:45.214596 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6cc5c-3708-48ef-91dd-30cb0d52df23 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:45.224568 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1405/2800] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:45.499754 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcc7ad-61e5-4ce9-aaf4-a4326d116135 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:45.501226 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a4fcc7ad-61e5-4ce9-aaf4-a4326d116135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:45.513100 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcc7ad-61e5-4ce9-aaf4-a4326d116135 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:45.517651 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:45.518286 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:45.518558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:45.518854 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e admin admin] PATH_INFO: `/v3/projects/cf3a11b11545423f8cc9e235e4acd31e` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:45.520514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf3a11b11545423f8cc9e235e4acd31e)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:45.521853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:45.568678 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf3a11b11545423f8cc9e235e4acd31e', 'request_id': 'req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:45.576827 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf3a11b11545423f8cc9e235e4acd31e', 'request_id': 'req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:45.583121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf3a11b11545423f8cc9e235e4acd31e', 'request_id': 'req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:45.594030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e admin admin] The token cache is being invalidate because project cf3a11b11545423f8cc9e235e4acd31e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:45.594314 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:45.594984 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:45.596456 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1396/2801] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:45 2022] DELETE /identity/v3/projects/cf3a11b11545423f8cc9e235e4acd31e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:45.600457 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6cc5c-3708-48ef-91dd-30cb0d52df23 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:45.611620 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6cc5c-3708-48ef-91dd-30cb0d52df23 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:45.630811 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:45.672937 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2038b1-3311-4531-9106-efc510599c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:45.673571 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2038b1-3311-4531-9106-efc510599c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:45.673850 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2038b1-3311-4531-9106-efc510599c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:45.674239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2038b1-3311-4531-9106-efc510599c08 admin admin] PATH_INFO: `/v3/users/b47e8e6bde284a5997a74b9a53f8b8f1` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:45.675455 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2038b1-3311-4531-9106-efc510599c08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b47e8e6bde284a5997a74b9a53f8b8f1)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:45.677301 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2038b1-3311-4531-9106-efc510599c08 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:45.723748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1d2038b1-3311-4531-9106-efc510599c08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b47e8e6bde284a5997a74b9a53f8b8f1'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:45.730810 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1d2038b1-3311-4531-9106-efc510599c08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b47e8e6bde284a5997a74b9a53f8b8f1'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:45.738284 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1d2038b1-3311-4531-9106-efc510599c08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b47e8e6bde284a5997a74b9a53f8b8f1'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:45.749900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1d2038b1-3311-4531-9106-efc510599c08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b47e8e6bde284a5997a74b9a53f8b8f1'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:45.750537 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-1d2038b1-3311-4531-9106-efc510599c08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b47e8e6bde284a5997a74b9a53f8b8f1'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:45.756292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1406/2802] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:45 2022] DELETE /identity/v3/users/b47e8e6bde284a5997a74b9a53f8b8f1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:46.029121 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:46.030662 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:46.044222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2e2d6b-9ac3-462d-b3d4-b938096f825e admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:46.108399 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3b6e7-afe2-455e-bdbf-8bf38ed864df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:46.109233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3b6e7-afe2-455e-bdbf-8bf38ed864df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:46.109515 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3b6e7-afe2-455e-bdbf-8bf38ed864df admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:46.109789 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3b6e7-afe2-455e-bdbf-8bf38ed864df admin admin] PATH_INFO: `/v3/projects/2ae85cd9d0724afdb34647fb11c094f2` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:46.110953 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3b6e7-afe2-455e-bdbf-8bf38ed864df admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ae85cd9d0724afdb34647fb11c094f2)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:46.112876 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f3b6e7-afe2-455e-bdbf-8bf38ed864df admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:46.146235 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:46.165518 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e7f3b6e7-afe2-455e-bdbf-8bf38ed864df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ae85cd9d0724afdb34647fb11c094f2', 'request_id': 'req-e7f3b6e7-afe2-455e-bdbf-8bf38ed864df'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:46.172871 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e7f3b6e7-afe2-455e-bdbf-8bf38ed864df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ae85cd9d0724afdb34647fb11c094f2', 'request_id': 'req-e7f3b6e7-afe2-455e-bdbf-8bf38ed864df'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:46.179583 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e7f3b6e7-afe2-455e-bdbf-8bf38ed864df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ae85cd9d0724afdb34647fb11c094f2', 'request_id': 'req-e7f3b6e7-afe2-455e-bdbf-8bf38ed864df'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:46.190288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e7f3b6e7-afe2-455e-bdbf-8bf38ed864df admin admin] The token cache is being invalidate because project 2ae85cd9d0724afdb34647fb11c094f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:46.190578 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e7f3b6e7-afe2-455e-bdbf-8bf38ed864df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:46.191237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-e7f3b6e7-afe2-455e-bdbf-8bf38ed864df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:46.192716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1397/2803] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:46 2022] DELETE /identity/v3/projects/2ae85cd9d0724afdb34647fb11c094f2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:46.197533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e107b9a-7426-46d7-ac38-a22f85921e15 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:46.198037 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e107b9a-7426-46d7-ac38-a22f85921e15 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:46.198135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e107b9a-7426-46d7-ac38-a22f85921e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:46.218239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-7e107b9a-7426-46d7-ac38-a22f85921e15 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:46.219403 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7e107b9a-7426-46d7-ac38-a22f85921e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:46.278757 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1407/2804] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:46.288521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2b7eb-b7cd-4e55-b518-393be4c8428c None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:46.288834 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2b7eb-b7cd-4e55-b518-393be4c8428c None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:46.289111 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2b7eb-b7cd-4e55-b518-393be4c8428c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:46.308750 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-6ac2b7eb-b7cd-4e55-b518-393be4c8428c None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:46.309978 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-6ac2b7eb-b7cd-4e55-b518-393be4c8428c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:46.318704 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1398/2805] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:46.322866 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e107b9a-7426-46d7-ac38-a22f85921e15 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:46.333325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e107b9a-7426-46d7-ac38-a22f85921e15 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:46.338153 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360708c-2397-42c1-b344-6227c150906e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:46.338856 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360708c-2397-42c1-b344-6227c150906e admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:46.339057 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360708c-2397-42c1-b344-6227c150906e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:46.339429 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360708c-2397-42c1-b344-6227c150906e admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:46.339961 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360708c-2397-42c1-b344-6227c150906e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:06:46.340264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360708c-2397-42c1-b344-6227c150906e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:46.341846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360708c-2397-42c1-b344-6227c150906e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:46.353390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1408/2806] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:06:46.357716 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2b7eb-b7cd-4e55-b518-393be4c8428c None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:46.367042 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2b7eb-b7cd-4e55-b518-393be4c8428c None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:46.371737 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfe7c5-69db-48a6-9d9e-168aaa0738f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:46.372326 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfe7c5-69db-48a6-9d9e-168aaa0738f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:46.372603 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfe7c5-69db-48a6-9d9e-168aaa0738f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:46.372871 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfe7c5-69db-48a6-9d9e-168aaa0738f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:46.373506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfe7c5-69db-48a6-9d9e-168aaa0738f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:46.376013 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfe7c5-69db-48a6-9d9e-168aaa0738f4 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:46.403589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1399/2807] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:46 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:46.407734 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360708c-2397-42c1-b344-6227c150906e admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:46.417720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360708c-2397-42c1-b344-6227c150906e admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:46.443294 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:46.476067 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ec73b-4015-462d-b149-fb7db0b830a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:46.476599 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ec73b-4015-462d-b149-fb7db0b830a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:46.476880 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ec73b-4015-462d-b149-fb7db0b830a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:46.477152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ec73b-4015-462d-b149-fb7db0b830a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:46.477790 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ec73b-4015-462d-b149-fb7db0b830a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:46.479720 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ec73b-4015-462d-b149-fb7db0b830a3 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:46.500938 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1409/2808] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:46 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:46.505088 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfe7c5-69db-48a6-9d9e-168aaa0738f4 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:46.516108 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfe7c5-69db-48a6-9d9e-168aaa0738f4 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:46.520698 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89ca9c-8678-49c5-8273-9dde4de7ce84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:46.521289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89ca9c-8678-49c5-8273-9dde4de7ce84 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:46.521568 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89ca9c-8678-49c5-8273-9dde4de7ce84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:46.521841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89ca9c-8678-49c5-8273-9dde4de7ce84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:46.522551 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89ca9c-8678-49c5-8273-9dde4de7ce84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:46.524170 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89ca9c-8678-49c5-8273-9dde4de7ce84 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:46.534052 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1400/2809] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:46.537809 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ec73b-4015-462d-b149-fb7db0b830a3 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:46.548087 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ec73b-4015-462d-b149-fb7db0b830a3 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:46.557445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de7dec-2aae-4299-a5ce-3a5f3b0ce09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:46.558131 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de7dec-2aae-4299-a5ce-3a5f3b0ce09d admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:46.558546 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de7dec-2aae-4299-a5ce-3a5f3b0ce09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:46.558861 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de7dec-2aae-4299-a5ce-3a5f3b0ce09d admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:46.559658 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de7dec-2aae-4299-a5ce-3a5f3b0ce09d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:46.561264 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de7dec-2aae-4299-a5ce-3a5f3b0ce09d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:46.580810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89ca9c-8678-49c5-8273-9dde4de7ce84 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:46.583239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1410/2810] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:46.593683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89ca9c-8678-49c5-8273-9dde4de7ce84 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:46.598498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1d3cc-1bae-4029-8b60-3cef40f46b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:46.599239 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1d3cc-1bae-4029-8b60-3cef40f46b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:46.599520 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1d3cc-1bae-4029-8b60-3cef40f46b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:46.599989 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1d3cc-1bae-4029-8b60-3cef40f46b99 admin admin] PATH_INFO: `/v3/projects/1da9b55393684dac9937e503cc58b00a/users/1d57a306bcf546c1979be27210114dd2/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:46.615357 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1d3cc-1bae-4029-8b60-3cef40f46b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da9b55393684dac9937e503cc58b00a, user_id=1d57a306bcf546c1979be27210114dd2, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:46.618058 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1d3cc-1bae-4029-8b60-3cef40f46b99 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:46.633098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1401/2811] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:46 2022] PUT /identity/v3/projects/1da9b55393684dac9937e503cc58b00a/users/1d57a306bcf546c1979be27210114dd2/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:46.643292 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a913c2d-ede9-4a10-8682-d16908f0b9d8 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:46.643814 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a913c2d-ede9-4a10-8682-d16908f0b9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:46.644375 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a913c2d-ede9-4a10-8682-d16908f0b9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:46.680083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-1a913c2d-ede9-4a10-8682-d16908f0b9d8 None None] MFA Rules not processed for user `1d57a306bcf546c1979be27210114dd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:46.681122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1a913c2d-ede9-4a10-8682-d16908f0b9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:46.766910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1411/2812] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:46.769274 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1d3cc-1bae-4029-8b60-3cef40f46b99 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:46.856118 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1d3cc-1bae-4029-8b60-3cef40f46b99 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:46.861315 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd4c56-026a-4205-98e6-816b4008b3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:46.862434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd4c56-026a-4205-98e6-816b4008b3f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:46.862434 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd4c56-026a-4205-98e6-816b4008b3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:46.862754 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd4c56-026a-4205-98e6-816b4008b3f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:46.863744 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd4c56-026a-4205-98e6-816b4008b3f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:46.865840 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd4c56-026a-4205-98e6-816b4008b3f9 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:46.899818 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1402/2813] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:46 2022] POST /identity/v3/projects => generated 367 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:46.903097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a913c2d-ede9-4a10-8682-d16908f0b9d8 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:46.916906 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a913c2d-ede9-4a10-8682-d16908f0b9d8 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:46.975654 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7e664-fd0b-42d4-afae-86ab274168a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:46.976449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7e664-fd0b-42d4-afae-86ab274168a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:46.976864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7e664-fd0b-42d4-afae-86ab274168a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:46.977193 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7e664-fd0b-42d4-afae-86ab274168a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:46.978202 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7e664-fd0b-42d4-afae-86ab274168a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:46.980159 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7e664-fd0b-42d4-afae-86ab274168a5 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:47.000952 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1412/2814] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:46 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:47.005086 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd4c56-026a-4205-98e6-816b4008b3f9 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:47.027002 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd4c56-026a-4205-98e6-816b4008b3f9 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:47.027002 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797c4a2-176f-4cd2-9e58-4255bde423f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:47.027002 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797c4a2-176f-4cd2-9e58-4255bde423f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:47.027002 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797c4a2-176f-4cd2-9e58-4255bde423f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:47.027388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797c4a2-176f-4cd2-9e58-4255bde423f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:47.027388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797c4a2-176f-4cd2-9e58-4255bde423f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:47.027388 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797c4a2-176f-4cd2-9e58-4255bde423f5 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:47.040963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1403/2815] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 15:06:47.042135 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7e664-fd0b-42d4-afae-86ab274168a5 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:47.053160 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7e664-fd0b-42d4-afae-86ab274168a5 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:47.058604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0baf5-7b8e-4ea2-946a-4f287be44b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:47.059311 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0baf5-7b8e-4ea2-946a-4f287be44b2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:47.059584 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0baf5-7b8e-4ea2-946a-4f287be44b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:47.059852 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d0baf5-7b8e-4ea2-946a-4f287be44b2d admin admin] PATH_INFO: `/v3/projects/237993b00c32443a98fabc92c5670f0a/users/967ccd8bef444f60a3535c2b37afef61/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:47.087014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0baf5-7b8e-4ea2-946a-4f287be44b2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=237993b00c32443a98fabc92c5670f0a, user_id=967ccd8bef444f60a3535c2b37afef61, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:47.087014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d0baf5-7b8e-4ea2-946a-4f287be44b2d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:47.098267 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1413/2816] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:47 2022] PUT /identity/v3/projects/237993b00c32443a98fabc92c5670f0a/users/967ccd8bef444f60a3535c2b37afef61/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:47.103568 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797c4a2-176f-4cd2-9e58-4255bde423f5 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:47.115099 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797c4a2-176f-4cd2-9e58-4255bde423f5 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:47.136025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:47.176011 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea89933f-9ad0-4382-ac70-589d3def6e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:47.176621 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89933f-9ad0-4382-ac70-589d3def6e0d admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:47.176902 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89933f-9ad0-4382-ac70-589d3def6e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:47.177174 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea89933f-9ad0-4382-ac70-589d3def6e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:47.177816 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89933f-9ad0-4382-ac70-589d3def6e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:47.179442 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea89933f-9ad0-4382-ac70-589d3def6e0d admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:47.189684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1404/2817] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:47.194028 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0baf5-7b8e-4ea2-946a-4f287be44b2d admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:47.199576 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:47.205759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d0baf5-7b8e-4ea2-946a-4f287be44b2d admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:47.211202 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055db2e7-8424-4f66-a95c-194abdea10dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:47.211876 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055db2e7-8424-4f66-a95c-194abdea10dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:47.212136 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055db2e7-8424-4f66-a95c-194abdea10dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:47.212414 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055db2e7-8424-4f66-a95c-194abdea10dd admin admin] PATH_INFO: `/v3/projects/237993b00c32443a98fabc92c5670f0a/users/967ccd8bef444f60a3535c2b37afef61/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:47.214417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055db2e7-8424-4f66-a95c-194abdea10dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=237993b00c32443a98fabc92c5670f0a, user_id=967ccd8bef444f60a3535c2b37afef61, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:47.217070 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055db2e7-8424-4f66-a95c-194abdea10dd admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:47.227844 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1414/2818] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:47 2022] PUT /identity/v3/projects/237993b00c32443a98fabc92c5670f0a/users/967ccd8bef444f60a3535c2b37afef61/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:06:47.239004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17740c3-dd93-45c8-bdc6-98a5115e7033 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:47.239295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17740c3-dd93-45c8-bdc6-98a5115e7033 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:47.239575 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17740c3-dd93-45c8-bdc6-98a5115e7033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:47.267318 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-d17740c3-dd93-45c8-bdc6-98a5115e7033 None None] MFA Rules not processed for user `967ccd8bef444f60a3535c2b37afef61`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:47.268523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d17740c3-dd93-45c8-bdc6-98a5115e7033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:47.339335 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1405/2819] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:47.351563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055db2e7-8424-4f66-a95c-194abdea10dd admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:47.353256 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-055db2e7-8424-4f66-a95c-194abdea10dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:47.430621 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055db2e7-8424-4f66-a95c-194abdea10dd admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:47.434403 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ae799-a1a7-403b-ac99-4ded4bc4cc77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:47.435045 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ae799-a1a7-403b-ac99-4ded4bc4cc77 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:47.435313 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ae799-a1a7-403b-ac99-4ded4bc4cc77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:47.435578 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ae799-a1a7-403b-ac99-4ded4bc4cc77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:47.444357 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ae799-a1a7-403b-ac99-4ded4bc4cc77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:47.445768 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ae799-a1a7-403b-ac99-4ded4bc4cc77 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:47.455417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1415/2820] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:47.499535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17740c3-dd93-45c8-bdc6-98a5115e7033 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:47.501032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d17740c3-dd93-45c8-bdc6-98a5115e7033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:47.512995 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17740c3-dd93-45c8-bdc6-98a5115e7033 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:47.517233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38de59c-eef8-461d-95f7-165597bc51e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:47.517911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38de59c-eef8-461d-95f7-165597bc51e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:47.518139 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38de59c-eef8-461d-95f7-165597bc51e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:47.518412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38de59c-eef8-461d-95f7-165597bc51e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:47.575311 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38de59c-eef8-461d-95f7-165597bc51e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:47.576752 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38de59c-eef8-461d-95f7-165597bc51e2 service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:47.587420 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1406/2821] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:50.948401 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ae799-a1a7-403b-ac99-4ded4bc4cc77 service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:51.010302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ae799-a1a7-403b-ac99-4ded4bc4cc77 service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:51.015104 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63cd59-e10f-4e06-a657-edfcbc66e4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:51.015724 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63cd59-e10f-4e06-a657-edfcbc66e4c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:51.015999 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63cd59-e10f-4e06-a657-edfcbc66e4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:51.016267 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63cd59-e10f-4e06-a657-edfcbc66e4c3 admin admin] PATH_INFO: `/v3/users/1d57a306bcf546c1979be27210114dd2` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:51.017367 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63cd59-e10f-4e06-a657-edfcbc66e4c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d57a306bcf546c1979be27210114dd2)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:51.019308 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63cd59-e10f-4e06-a657-edfcbc66e4c3 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:51.065721 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-9a63cd59-e10f-4e06-a657-edfcbc66e4c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d57a306bcf546c1979be27210114dd2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:51.073342 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-9a63cd59-e10f-4e06-a657-edfcbc66e4c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d57a306bcf546c1979be27210114dd2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:51.080833 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-9a63cd59-e10f-4e06-a657-edfcbc66e4c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d57a306bcf546c1979be27210114dd2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:51.089591 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:51.095117 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-9a63cd59-e10f-4e06-a657-edfcbc66e4c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d57a306bcf546c1979be27210114dd2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:51.095944 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-9a63cd59-e10f-4e06-a657-edfcbc66e4c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d57a306bcf546c1979be27210114dd2'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:51.102462 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1416/2822] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:50 2022] DELETE /identity/v3/users/1d57a306bcf546c1979be27210114dd2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:51.106968 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feca2a-50d3-4518-8492-c8df37169c5b None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:51.107446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feca2a-50d3-4518-8492-c8df37169c5b None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:51.107565 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08feca2a-50d3-4518-8492-c8df37169c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:51.126352 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-08feca2a-50d3-4518-8492-c8df37169c5b None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:51.127598 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-08feca2a-50d3-4518-8492-c8df37169c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:51.167861 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=100 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:51.186720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1407/2823] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:51.196591 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63cd59-e10f-4e06-a657-edfcbc66e4c3 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:51.197998 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-9a63cd59-e10f-4e06-a657-edfcbc66e4c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:51.258588 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63cd59-e10f-4e06-a657-edfcbc66e4c3 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:51.261941 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea9af7-e1ae-487f-b513-4b1f8b9cd86f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:51.262540 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea9af7-e1ae-487f-b513-4b1f8b9cd86f service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:51.262840 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea9af7-e1ae-487f-b513-4b1f8b9cd86f service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:51.263108 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea9af7-e1ae-487f-b513-4b1f8b9cd86f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:51.272785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea9af7-e1ae-487f-b513-4b1f8b9cd86f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:51.274252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea9af7-e1ae-487f-b513-4b1f8b9cd86f service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:51.284846 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1417/2824] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:51.550680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08feca2a-50d3-4518-8492-c8df37169c5b None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:51.552145 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-08feca2a-50d3-4518-8492-c8df37169c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:51.565280 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08feca2a-50d3-4518-8492-c8df37169c5b None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:51.570349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25795b-b31d-446a-a02b-e1044931167c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:51.571070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25795b-b31d-446a-a02b-e1044931167c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:51.571379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25795b-b31d-446a-a02b-e1044931167c admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:51.571636 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25795b-b31d-446a-a02b-e1044931167c admin admin] PATH_INFO: `/v3/projects/1da9b55393684dac9937e503cc58b00a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:51.572760 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25795b-b31d-446a-a02b-e1044931167c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1da9b55393684dac9937e503cc58b00a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:51.574654 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25795b-b31d-446a-a02b-e1044931167c admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:51.626722 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-cf25795b-b31d-446a-a02b-e1044931167c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1da9b55393684dac9937e503cc58b00a', 'request_id': 'req-cf25795b-b31d-446a-a02b-e1044931167c'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:51.635003 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-cf25795b-b31d-446a-a02b-e1044931167c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1da9b55393684dac9937e503cc58b00a', 'request_id': 'req-cf25795b-b31d-446a-a02b-e1044931167c'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:51.643566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-cf25795b-b31d-446a-a02b-e1044931167c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1da9b55393684dac9937e503cc58b00a', 'request_id': 'req-cf25795b-b31d-446a-a02b-e1044931167c'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:51.653828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-cf25795b-b31d-446a-a02b-e1044931167c admin admin] The token cache is being invalidate because project 1da9b55393684dac9937e503cc58b00a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:51.654112 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-cf25795b-b31d-446a-a02b-e1044931167c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:51.654807 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-cf25795b-b31d-446a-a02b-e1044931167c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:51.656415 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1408/2825] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:51 2022] DELETE /identity/v3/projects/1da9b55393684dac9937e503cc58b00a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:51.660515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea9af7-e1ae-487f-b513-4b1f8b9cd86f service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:51.670604 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea9af7-e1ae-487f-b513-4b1f8b9cd86f service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:51.728252 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765aa66-56b2-42db-92b3-96def29a5e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:51.728874 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765aa66-56b2-42db-92b3-96def29a5e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:51.729148 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765aa66-56b2-42db-92b3-96def29a5e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:51.729414 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765aa66-56b2-42db-92b3-96def29a5e3c admin admin] PATH_INFO: `/v3/users/967ccd8bef444f60a3535c2b37afef61` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:51.730503 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765aa66-56b2-42db-92b3-96def29a5e3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=967ccd8bef444f60a3535c2b37afef61)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:51.732430 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765aa66-56b2-42db-92b3-96def29a5e3c admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:51.777644 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0765aa66-56b2-42db-92b3-96def29a5e3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '967ccd8bef444f60a3535c2b37afef61'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:51.785129 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0765aa66-56b2-42db-92b3-96def29a5e3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '967ccd8bef444f60a3535c2b37afef61'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:51.792280 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0765aa66-56b2-42db-92b3-96def29a5e3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '967ccd8bef444f60a3535c2b37afef61'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:51.804622 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0765aa66-56b2-42db-92b3-96def29a5e3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '967ccd8bef444f60a3535c2b37afef61'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:51.805205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-0765aa66-56b2-42db-92b3-96def29a5e3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '967ccd8bef444f60a3535c2b37afef61'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:51.811027 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1418/2826] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:51 2022] DELETE /identity/v3/users/967ccd8bef444f60a3535c2b37afef61 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:52.095860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25795b-b31d-446a-a02b-e1044931167c admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:52.097476 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-cf25795b-b31d-446a-a02b-e1044931167c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:52.115490 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25795b-b31d-446a-a02b-e1044931167c admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:52.176089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ff45f-fbf5-4d36-8f2b-5ec93fc9653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:52.176717 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ff45f-fbf5-4d36-8f2b-5ec93fc9653a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:52.176993 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ff45f-fbf5-4d36-8f2b-5ec93fc9653a admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:52.177265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ff45f-fbf5-4d36-8f2b-5ec93fc9653a admin admin] PATH_INFO: `/v3/projects/237993b00c32443a98fabc92c5670f0a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:52.178376 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ff45f-fbf5-4d36-8f2b-5ec93fc9653a admin admin] RBAC: Authorizing `identity:delete_project(project_id=237993b00c32443a98fabc92c5670f0a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:52.180306 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ff45f-fbf5-4d36-8f2b-5ec93fc9653a admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:52.226962 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f35ff45f-fbf5-4d36-8f2b-5ec93fc9653a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '237993b00c32443a98fabc92c5670f0a', 'request_id': 'req-f35ff45f-fbf5-4d36-8f2b-5ec93fc9653a'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:52.235016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f35ff45f-fbf5-4d36-8f2b-5ec93fc9653a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '237993b00c32443a98fabc92c5670f0a', 'request_id': 'req-f35ff45f-fbf5-4d36-8f2b-5ec93fc9653a'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:52.242336 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f35ff45f-fbf5-4d36-8f2b-5ec93fc9653a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '237993b00c32443a98fabc92c5670f0a', 'request_id': 'req-f35ff45f-fbf5-4d36-8f2b-5ec93fc9653a'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:52.250173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:52.255599 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f35ff45f-fbf5-4d36-8f2b-5ec93fc9653a admin admin] The token cache is being invalidate because project 237993b00c32443a98fabc92c5670f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:52.256163 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f35ff45f-fbf5-4d36-8f2b-5ec93fc9653a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:52.257173 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-f35ff45f-fbf5-4d36-8f2b-5ec93fc9653a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:52.258934 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1409/2827] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:52 2022] DELETE /identity/v3/projects/237993b00c32443a98fabc92c5670f0a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:52.263952 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754996db-e48a-423b-96df-0c768dacf689 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:52.264417 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754996db-e48a-423b-96df-0c768dacf689 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:52.264534 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754996db-e48a-423b-96df-0c768dacf689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:52.284131 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-754996db-e48a-423b-96df-0c768dacf689 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:52.285335 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-754996db-e48a-423b-96df-0c768dacf689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:52.294239 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:52.350793 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1419/2828] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:06:52.415566 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3910efc-8de8-4641-ad7d-1f7391302ee6 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:52.415843 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3910efc-8de8-4641-ad7d-1f7391302ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:52.416114 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3910efc-8de8-4641-ad7d-1f7391302ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:52.437273 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-c3910efc-8de8-4641-ad7d-1f7391302ee6 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:52.438673 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-c3910efc-8de8-4641-ad7d-1f7391302ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:52.446686 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1410/2829] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:52.451225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754996db-e48a-423b-96df-0c768dacf689 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:52.461231 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754996db-e48a-423b-96df-0c768dacf689 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:52.466054 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19262389-fb6a-4550-82b1-370388a5272b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:52.466656 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19262389-fb6a-4550-82b1-370388a5272b admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:52.467013 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19262389-fb6a-4550-82b1-370388a5272b admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:52.467287 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19262389-fb6a-4550-82b1-370388a5272b admin admin] PATH_INFO: `/v3/domains` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:52.467952 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19262389-fb6a-4550-82b1-370388a5272b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81454) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:06:52.468796 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19262389-fb6a-4550-82b1-370388a5272b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:52.470364 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19262389-fb6a-4550-82b1-370388a5272b admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:52.482375 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1420/2830] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:06:52.487625 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3910efc-8de8-4641-ad7d-1f7391302ee6 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:52.496701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3910efc-8de8-4641-ad7d-1f7391302ee6 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:52.502701 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935421f0-d942-4884-9b4f-3c3dcf7565e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:52.503339 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935421f0-d942-4884-9b4f-3c3dcf7565e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:52.503617 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935421f0-d942-4884-9b4f-3c3dcf7565e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:52.503887 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935421f0-d942-4884-9b4f-3c3dcf7565e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:52.504526 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935421f0-d942-4884-9b4f-3c3dcf7565e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:52.506444 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935421f0-d942-4884-9b4f-3c3dcf7565e4 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:52.545502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1411/2831] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:52 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:52.550542 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19262389-fb6a-4550-82b1-370388a5272b admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:52.561712 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19262389-fb6a-4550-82b1-370388a5272b admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:52.623338 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec832d-5739-4196-aecb-a78352802e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:52.623956 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec832d-5739-4196-aecb-a78352802e7a admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:52.624238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec832d-5739-4196-aecb-a78352802e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:52.624539 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec832d-5739-4196-aecb-a78352802e7a admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:52.625379 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec832d-5739-4196-aecb-a78352802e7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:52.627317 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec832d-5739-4196-aecb-a78352802e7a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:52.651528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1421/2832] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:52 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:52.655769 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935421f0-d942-4884-9b4f-3c3dcf7565e4 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:52.675092 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935421f0-d942-4884-9b4f-3c3dcf7565e4 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:52.681381 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80ee44-8bfe-4eb1-b1e4-858b38659558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:52.682004 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80ee44-8bfe-4eb1-b1e4-858b38659558 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:52.682282 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80ee44-8bfe-4eb1-b1e4-858b38659558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:52.682679 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80ee44-8bfe-4eb1-b1e4-858b38659558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:52.683226 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80ee44-8bfe-4eb1-b1e4-858b38659558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:52.684842 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80ee44-8bfe-4eb1-b1e4-858b38659558 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:52.696268 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1412/2833] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:52.700316 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec832d-5739-4196-aecb-a78352802e7a admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:52.711519 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec832d-5739-4196-aecb-a78352802e7a admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:52.716919 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e6f29-6355-47ae-aa00-04dfa9b758e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:52.717525 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e6f29-6355-47ae-aa00-04dfa9b758e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:52.717825 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e6f29-6355-47ae-aa00-04dfa9b758e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:52.718058 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e6f29-6355-47ae-aa00-04dfa9b758e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:52.719305 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e6f29-6355-47ae-aa00-04dfa9b758e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:52.720905 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e6f29-6355-47ae-aa00-04dfa9b758e6 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:52.732346 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1422/2834] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:52.737711 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80ee44-8bfe-4eb1-b1e4-858b38659558 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:52.748127 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80ee44-8bfe-4eb1-b1e4-858b38659558 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:52.752877 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e1fef-fcb4-4ec4-bcd3-52d81f7f788f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:52.753528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e1fef-fcb4-4ec4-bcd3-52d81f7f788f admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:52.753806 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e1fef-fcb4-4ec4-bcd3-52d81f7f788f admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:52.754078 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e1fef-fcb4-4ec4-bcd3-52d81f7f788f admin admin] PATH_INFO: `/v3/projects/bf46e9fab0f64a43a3c6ecea9c786751/users/a065844f98dd40a98f01dda98b434652/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:52.770486 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e1fef-fcb4-4ec4-bcd3-52d81f7f788f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf46e9fab0f64a43a3c6ecea9c786751, user_id=a065844f98dd40a98f01dda98b434652, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:52.773402 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e1fef-fcb4-4ec4-bcd3-52d81f7f788f admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:52.787586 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1413/2835] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:52 2022] PUT /identity/v3/projects/bf46e9fab0f64a43a3c6ecea9c786751/users/a065844f98dd40a98f01dda98b434652/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:52.798290 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda2084-3d2f-4767-a78f-e64b930f67ef None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:52.798572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda2084-3d2f-4767-a78f-e64b930f67ef None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:52.798868 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda2084-3d2f-4767-a78f-e64b930f67ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:52.828586 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-1cda2084-3d2f-4767-a78f-e64b930f67ef None None] MFA Rules not processed for user `a065844f98dd40a98f01dda98b434652`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:52.829811 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1cda2084-3d2f-4767-a78f-e64b930f67ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:52.902602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1423/2836] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:52.908126 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e1fef-fcb4-4ec4-bcd3-52d81f7f788f admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:52.980371 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e1fef-fcb4-4ec4-bcd3-52d81f7f788f admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:52.985480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d52711-7024-4bf9-a906-7bcdef1c14e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:52.986099 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d52711-7024-4bf9-a906-7bcdef1c14e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:52.986379 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d52711-7024-4bf9-a906-7bcdef1c14e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:52.986684 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d52711-7024-4bf9-a906-7bcdef1c14e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:52.987331 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d52711-7024-4bf9-a906-7bcdef1c14e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:52.989233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d52711-7024-4bf9-a906-7bcdef1c14e3 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:53.016254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1414/2837] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:52 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:53.020574 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda2084-3d2f-4767-a78f-e64b930f67ef None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:53.031180 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda2084-3d2f-4767-a78f-e64b930f67ef None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:53.035779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:53.090411 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089ca78-b788-4278-ad78-727fb3e8a461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:53.091063 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089ca78-b788-4278-ad78-727fb3e8a461 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:53.091372 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089ca78-b788-4278-ad78-727fb3e8a461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:53.091639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a089ca78-b788-4278-ad78-727fb3e8a461 admin admin] PATH_INFO: `/v3/users` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:53.092268 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089ca78-b788-4278-ad78-727fb3e8a461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:53.094229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a089ca78-b788-4278-ad78-727fb3e8a461 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:53.115318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1424/2838] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:53 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:53.119348 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d52711-7024-4bf9-a906-7bcdef1c14e3 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:53.130521 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d52711-7024-4bf9-a906-7bcdef1c14e3 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:53.135233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec0de7-d0b3-4e0a-8ee5-7b20cb68d714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:53.135829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec0de7-d0b3-4e0a-8ee5-7b20cb68d714 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:53.136105 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec0de7-d0b3-4e0a-8ee5-7b20cb68d714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:53.136375 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec0de7-d0b3-4e0a-8ee5-7b20cb68d714 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:53.137015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec0de7-d0b3-4e0a-8ee5-7b20cb68d714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:53.138592 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec0de7-d0b3-4e0a-8ee5-7b20cb68d714 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:53.149848 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1415/2839] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:53.154149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089ca78-b788-4278-ad78-727fb3e8a461 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:53.163902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a089ca78-b788-4278-ad78-727fb3e8a461 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:53.168440 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02d6be-4281-43be-9491-207c3ad792cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:53.169078 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02d6be-4281-43be-9491-207c3ad792cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:53.169352 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02d6be-4281-43be-9491-207c3ad792cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:53.169619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02d6be-4281-43be-9491-207c3ad792cc admin admin] PATH_INFO: `/v3/projects/c1fd476ef17e404b9571fde7c48199b4/users/0bb9f504d3fd4768ae31f688ceb7fa6e/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:53.185740 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02d6be-4281-43be-9491-207c3ad792cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1fd476ef17e404b9571fde7c48199b4, user_id=0bb9f504d3fd4768ae31f688ceb7fa6e, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:53.188938 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02d6be-4281-43be-9491-207c3ad792cc admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:53.203407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1425/2840] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:53 2022] PUT /identity/v3/projects/c1fd476ef17e404b9571fde7c48199b4/users/0bb9f504d3fd4768ae31f688ceb7fa6e/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:53.207439 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec0de7-d0b3-4e0a-8ee5-7b20cb68d714 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:53.217520 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec0de7-d0b3-4e0a-8ee5-7b20cb68d714 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:53.276025 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908920ce-8a9c-4361-b1ca-5659075fcb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:53.276660 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908920ce-8a9c-4361-b1ca-5659075fcb46 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:53.276939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908920ce-8a9c-4361-b1ca-5659075fcb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:53.277211 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908920ce-8a9c-4361-b1ca-5659075fcb46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:53.277847 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908920ce-8a9c-4361-b1ca-5659075fcb46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:53.279480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908920ce-8a9c-4361-b1ca-5659075fcb46 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:53.289202 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1416/2841] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:53.293319 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02d6be-4281-43be-9491-207c3ad792cc admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:53.304150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02d6be-4281-43be-9491-207c3ad792cc admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:53.309207 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229c6df-cdb0-4ffe-819b-2685825a4990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:53.309864 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229c6df-cdb0-4ffe-819b-2685825a4990 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:53.310150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229c6df-cdb0-4ffe-819b-2685825a4990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:53.310420 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229c6df-cdb0-4ffe-819b-2685825a4990 admin admin] PATH_INFO: `/v3/projects/c1fd476ef17e404b9571fde7c48199b4/users/0bb9f504d3fd4768ae31f688ceb7fa6e/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:53.313009 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229c6df-cdb0-4ffe-819b-2685825a4990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1fd476ef17e404b9571fde7c48199b4, user_id=0bb9f504d3fd4768ae31f688ceb7fa6e, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:53.315678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229c6df-cdb0-4ffe-819b-2685825a4990 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:53.326233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1426/2842] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:53 2022] PUT /identity/v3/projects/c1fd476ef17e404b9571fde7c48199b4/users/0bb9f504d3fd4768ae31f688ceb7fa6e/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:53.338420 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca26fb-ec35-40f2-9df1-389f47c56abb None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:53.338722 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca26fb-ec35-40f2-9df1-389f47c56abb None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:53.339017 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca26fb-ec35-40f2-9df1-389f47c56abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:53.347129 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:53.367722 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-94ca26fb-ec35-40f2-9df1-389f47c56abb None None] MFA Rules not processed for user `0bb9f504d3fd4768ae31f688ceb7fa6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:53.368939 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-94ca26fb-ec35-40f2-9df1-389f47c56abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:53.436643 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1417/2843] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:53.448734 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229c6df-cdb0-4ffe-819b-2685825a4990 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:53.450413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d229c6df-cdb0-4ffe-819b-2685825a4990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:53.509299 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229c6df-cdb0-4ffe-819b-2685825a4990 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:53.513789 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaafcc5-b24e-44ad-8f2e-838a20e8a667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:53.514393 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaafcc5-b24e-44ad-8f2e-838a20e8a667 service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:53.514671 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaafcc5-b24e-44ad-8f2e-838a20e8a667 service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:53.515012 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaafcc5-b24e-44ad-8f2e-838a20e8a667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:53.523996 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaafcc5-b24e-44ad-8f2e-838a20e8a667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:53.525411 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaafcc5-b24e-44ad-8f2e-838a20e8a667 service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:53.534906 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1427/2844] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:53.565045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca26fb-ec35-40f2-9df1-389f47c56abb None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:53.628062 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca26fb-ec35-40f2-9df1-389f47c56abb None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:53.633112 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3020084-a949-4d47-8c09-962bc5e217f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:53.633897 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3020084-a949-4d47-8c09-962bc5e217f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:53.634241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3020084-a949-4d47-8c09-962bc5e217f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:53.634559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3020084-a949-4d47-8c09-962bc5e217f5 admin admin] PATH_INFO: `/v3/users/a065844f98dd40a98f01dda98b434652` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:53.635873 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3020084-a949-4d47-8c09-962bc5e217f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a065844f98dd40a98f01dda98b434652)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:53.638098 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3020084-a949-4d47-8c09-962bc5e217f5 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:53.684386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-a3020084-a949-4d47-8c09-962bc5e217f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a065844f98dd40a98f01dda98b434652'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:53.691881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-a3020084-a949-4d47-8c09-962bc5e217f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a065844f98dd40a98f01dda98b434652'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:53.692589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-a3020084-a949-4d47-8c09-962bc5e217f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a065844f98dd40a98f01dda98b434652'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:53.704565 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-a3020084-a949-4d47-8c09-962bc5e217f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a065844f98dd40a98f01dda98b434652'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:53.705211 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-a3020084-a949-4d47-8c09-962bc5e217f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a065844f98dd40a98f01dda98b434652'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:53.717589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1418/2845] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:53 2022] DELETE /identity/v3/users/a065844f98dd40a98f01dda98b434652 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:53.722706 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd347447-14c7-40e8-8695-ad4d6976b995 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:53.723021 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd347447-14c7-40e8-8695-ad4d6976b995 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:53.723295 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd347447-14c7-40e8-8695-ad4d6976b995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:53.743718 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-cd347447-14c7-40e8-8695-ad4d6976b995 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:53.744949 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-cd347447-14c7-40e8-8695-ad4d6976b995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:53.809349 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1428/2846] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:53.820348 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3020084-a949-4d47-8c09-962bc5e217f5 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:53.821852 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a3020084-a949-4d47-8c09-962bc5e217f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:53.833764 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:53.889580 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3020084-a949-4d47-8c09-962bc5e217f5 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:53.893350 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90fc09-7a8f-42f5-81cb-80b3d0f2afaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:53.893969 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90fc09-7a8f-42f5-81cb-80b3d0f2afaa service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:53.894247 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90fc09-7a8f-42f5-81cb-80b3d0f2afaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:53.894946 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90fc09-7a8f-42f5-81cb-80b3d0f2afaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:53.904683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90fc09-7a8f-42f5-81cb-80b3d0f2afaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:53.906650 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90fc09-7a8f-42f5-81cb-80b3d0f2afaa service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:53.917307 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1419/2847] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:06:54.214368 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd347447-14c7-40e8-8695-ad4d6976b995 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:54.215965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-cd347447-14c7-40e8-8695-ad4d6976b995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:54.227985 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:54.236813 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd347447-14c7-40e8-8695-ad4d6976b995 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:54.252766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ea165-e629-4313-a982-69122276423d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:54.254050 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ea165-e629-4313-a982-69122276423d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:54.254349 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ea165-e629-4313-a982-69122276423d admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:54.255281 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208ea165-e629-4313-a982-69122276423d admin admin] PATH_INFO: `/v3/projects/bf46e9fab0f64a43a3c6ecea9c786751` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:54.259572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ea165-e629-4313-a982-69122276423d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf46e9fab0f64a43a3c6ecea9c786751)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:54.261737 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208ea165-e629-4313-a982-69122276423d admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:54.316009 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-208ea165-e629-4313-a982-69122276423d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf46e9fab0f64a43a3c6ecea9c786751', 'request_id': 'req-208ea165-e629-4313-a982-69122276423d'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.322305 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-208ea165-e629-4313-a982-69122276423d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf46e9fab0f64a43a3c6ecea9c786751', 'request_id': 'req-208ea165-e629-4313-a982-69122276423d'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.329367 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-208ea165-e629-4313-a982-69122276423d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf46e9fab0f64a43a3c6ecea9c786751', 'request_id': 'req-208ea165-e629-4313-a982-69122276423d'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.341689 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-208ea165-e629-4313-a982-69122276423d admin admin] The token cache is being invalidate because project bf46e9fab0f64a43a3c6ecea9c786751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:54.341976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-208ea165-e629-4313-a982-69122276423d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.342614 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-208ea165-e629-4313-a982-69122276423d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.344871 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1429/2848] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:54 2022] DELETE /identity/v3/projects/bf46e9fab0f64a43a3c6ecea9c786751 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:54.349103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90fc09-7a8f-42f5-81cb-80b3d0f2afaa service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:54.358634 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90fc09-7a8f-42f5-81cb-80b3d0f2afaa service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:54.415646 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f8dab-e294-4a65-b595-9ea688b0ed44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:54.416271 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f8dab-e294-4a65-b595-9ea688b0ed44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:54.416550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f8dab-e294-4a65-b595-9ea688b0ed44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:54.416823 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f8dab-e294-4a65-b595-9ea688b0ed44 admin admin] PATH_INFO: `/v3/users/0bb9f504d3fd4768ae31f688ceb7fa6e` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:54.417940 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f8dab-e294-4a65-b595-9ea688b0ed44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bb9f504d3fd4768ae31f688ceb7fa6e)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:54.419885 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f8dab-e294-4a65-b595-9ea688b0ed44 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:54.467834 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-752f8dab-e294-4a65-b595-9ea688b0ed44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bb9f504d3fd4768ae31f688ceb7fa6e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.474991 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-752f8dab-e294-4a65-b595-9ea688b0ed44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bb9f504d3fd4768ae31f688ceb7fa6e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.475610 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-752f8dab-e294-4a65-b595-9ea688b0ed44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bb9f504d3fd4768ae31f688ceb7fa6e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.487657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-752f8dab-e294-4a65-b595-9ea688b0ed44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bb9f504d3fd4768ae31f688ceb7fa6e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.488328 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-752f8dab-e294-4a65-b595-9ea688b0ed44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bb9f504d3fd4768ae31f688ceb7fa6e'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.501771 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1420/2849] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:06:54 2022] DELETE /identity/v3/users/0bb9f504d3fd4768ae31f688ceb7fa6e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:54.780690 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ea165-e629-4313-a982-69122276423d admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:54.782146 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-208ea165-e629-4313-a982-69122276423d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:54.795047 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208ea165-e629-4313-a982-69122276423d admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:54.853289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcebab4a-ca5e-4d4d-8e46-8c977e93b60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:54.853929 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebab4a-ca5e-4d4d-8e46-8c977e93b60a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:54.854205 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebab4a-ca5e-4d4d-8e46-8c977e93b60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:54.854479 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebab4a-ca5e-4d4d-8e46-8c977e93b60a admin admin] PATH_INFO: `/v3/projects/c1fd476ef17e404b9571fde7c48199b4` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:54.855637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcebab4a-ca5e-4d4d-8e46-8c977e93b60a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1fd476ef17e404b9571fde7c48199b4)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:54.857544 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcebab4a-ca5e-4d4d-8e46-8c977e93b60a admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:54.904285 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-bcebab4a-ca5e-4d4d-8e46-8c977e93b60a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1fd476ef17e404b9571fde7c48199b4', 'request_id': 'req-bcebab4a-ca5e-4d4d-8e46-8c977e93b60a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.910325 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-bcebab4a-ca5e-4d4d-8e46-8c977e93b60a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1fd476ef17e404b9571fde7c48199b4', 'request_id': 'req-bcebab4a-ca5e-4d4d-8e46-8c977e93b60a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.915804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-bcebab4a-ca5e-4d4d-8e46-8c977e93b60a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1fd476ef17e404b9571fde7c48199b4', 'request_id': 'req-bcebab4a-ca5e-4d4d-8e46-8c977e93b60a'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.928121 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-bcebab4a-ca5e-4d4d-8e46-8c977e93b60a admin admin] The token cache is being invalidate because project c1fd476ef17e404b9571fde7c48199b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:06:54.928453 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-bcebab4a-ca5e-4d4d-8e46-8c977e93b60a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.929055 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-bcebab4a-ca5e-4d4d-8e46-8c977e93b60a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:06:54.930528 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1430/2850] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:06:54 2022] DELETE /identity/v3/projects/c1fd476ef17e404b9571fde7c48199b4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:54.935536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b26bf1-af87-4e38-8c71-8425275fd941 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:54.935931 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b26bf1-af87-4e38-8c71-8425275fd941 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:54.936214 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b26bf1-af87-4e38-8c71-8425275fd941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:54.956398 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-87b26bf1-af87-4e38-8c71-8425275fd941 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:54.957598 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-87b26bf1-af87-4e38-8c71-8425275fd941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:54.983221 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:55.026821 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1421/2851] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:55.034583 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0a330-8abb-4d98-a985-5188cc4753c6 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.035045 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0a330-8abb-4d98-a985-5188cc4753c6 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.035318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0a330-8abb-4d98-a985-5188cc4753c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.055074 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-afc0a330-8abb-4d98-a985-5188cc4753c6 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:55.056276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-afc0a330-8abb-4d98-a985-5188cc4753c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:55.064624 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1431/2852] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:55.068863 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b26bf1-af87-4e38-8c71-8425275fd941 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.078583 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b26bf1-af87-4e38-8c71-8425275fd941 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.083540 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049febf2-d091-4ef1-8881-4166b8f40acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.083792 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049febf2-d091-4ef1-8881-4166b8f40acb admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.084324 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049febf2-d091-4ef1-8881-4166b8f40acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.084592 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049febf2-d091-4ef1-8881-4166b8f40acb admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.085233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049febf2-d091-4ef1-8881-4166b8f40acb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:06:55.085563 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049febf2-d091-4ef1-8881-4166b8f40acb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.087147 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049febf2-d091-4ef1-8881-4166b8f40acb admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.099302 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1422/2853] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 15:06:55.103952 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0a330-8abb-4d98-a985-5188cc4753c6 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.113639 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0a330-8abb-4d98-a985-5188cc4753c6 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.119119 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95928a9c-49d4-4663-b11b-cdce55634d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.119254 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95928a9c-49d4-4663-b11b-cdce55634d62 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.119439 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95928a9c-49d4-4663-b11b-cdce55634d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.119704 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95928a9c-49d4-4663-b11b-cdce55634d62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.120678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95928a9c-49d4-4663-b11b-cdce55634d62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.122361 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95928a9c-49d4-4663-b11b-cdce55634d62 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.133181 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:55.151975 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1432/2854] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:55 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:55.156647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049febf2-d091-4ef1-8881-4166b8f40acb admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.166498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049febf2-d091-4ef1-8881-4166b8f40acb admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.222051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a103827-6ac9-40c9-8a6c-2ea84f233648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.222650 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a103827-6ac9-40c9-8a6c-2ea84f233648 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.222956 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a103827-6ac9-40c9-8a6c-2ea84f233648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.223227 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a103827-6ac9-40c9-8a6c-2ea84f233648 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.223848 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a103827-6ac9-40c9-8a6c-2ea84f233648 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.225751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a103827-6ac9-40c9-8a6c-2ea84f233648 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.248294 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1423/2855] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:55 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:55.252404 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95928a9c-49d4-4663-b11b-cdce55634d62 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.262881 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95928a9c-49d4-4663-b11b-cdce55634d62 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.267560 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57010c-829b-4ae6-b610-4a5208d12b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.268091 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57010c-829b-4ae6-b610-4a5208d12b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.268367 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57010c-829b-4ae6-b610-4a5208d12b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.268636 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57010c-829b-4ae6-b610-4a5208d12b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.269269 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e57010c-829b-4ae6-b610-4a5208d12b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.270882 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e57010c-829b-4ae6-b610-4a5208d12b4e admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.281168 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1433/2856] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:55.285155 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a103827-6ac9-40c9-8a6c-2ea84f233648 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.297249 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a103827-6ac9-40c9-8a6c-2ea84f233648 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.303603 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33d6de-3992-44c9-86a7-be2f72c1664e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.304214 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33d6de-3992-44c9-86a7-be2f72c1664e admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.304491 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33d6de-3992-44c9-86a7-be2f72c1664e admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.304760 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33d6de-3992-44c9-86a7-be2f72c1664e admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.305405 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33d6de-3992-44c9-86a7-be2f72c1664e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.307275 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33d6de-3992-44c9-86a7-be2f72c1664e admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.319974 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1424/2857] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:55.330302 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57010c-829b-4ae6-b610-4a5208d12b4e admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.340282 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57010c-829b-4ae6-b610-4a5208d12b4e admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.345677 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3076f90-da3d-49bc-b15b-f9ddc7690ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.346414 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3076f90-da3d-49bc-b15b-f9ddc7690ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.346613 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3076f90-da3d-49bc-b15b-f9ddc7690ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.346921 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3076f90-da3d-49bc-b15b-f9ddc7690ee8 admin admin] PATH_INFO: `/v3/projects/6d8adb54c72c46c9a05e1668079d4d68/users/9aa7270ef26f415dad1e860723b94731/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.362420 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3076f90-da3d-49bc-b15b-f9ddc7690ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d8adb54c72c46c9a05e1668079d4d68, user_id=9aa7270ef26f415dad1e860723b94731, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.365097 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3076f90-da3d-49bc-b15b-f9ddc7690ee8 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.377997 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1434/2858] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:55 2022] PUT /identity/v3/projects/6d8adb54c72c46c9a05e1668079d4d68/users/9aa7270ef26f415dad1e860723b94731/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:55.388289 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78b6dc-6225-456a-99d3-28126f092fd1 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.388760 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78b6dc-6225-456a-99d3-28126f092fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.388879 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78b6dc-6225-456a-99d3-28126f092fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.418472 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-db78b6dc-6225-456a-99d3-28126f092fd1 None None] MFA Rules not processed for user `9aa7270ef26f415dad1e860723b94731`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:55.419731 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-db78b6dc-6225-456a-99d3-28126f092fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:55.483068 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1425/2859] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:55.488118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3076f90-da3d-49bc-b15b-f9ddc7690ee8 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.549896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3076f90-da3d-49bc-b15b-f9ddc7690ee8 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.554673 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f263fe-0199-4f0d-ab9a-fe9306f891a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.555333 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f263fe-0199-4f0d-ab9a-fe9306f891a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.555623 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f263fe-0199-4f0d-ab9a-fe9306f891a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.555898 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f263fe-0199-4f0d-ab9a-fe9306f891a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.556573 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f263fe-0199-4f0d-ab9a-fe9306f891a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.558472 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f263fe-0199-4f0d-ab9a-fe9306f891a5 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.590677 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1435/2860] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:06:55 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:06:55.596369 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78b6dc-6225-456a-99d3-28126f092fd1 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.608030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78b6dc-6225-456a-99d3-28126f092fd1 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.633641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:55.671608 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a2eda-62c8-443b-9ad0-1cac0bfb977c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.672233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a2eda-62c8-443b-9ad0-1cac0bfb977c admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.672512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a2eda-62c8-443b-9ad0-1cac0bfb977c admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.672804 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a2eda-62c8-443b-9ad0-1cac0bfb977c admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.673473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a2eda-62c8-443b-9ad0-1cac0bfb977c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.675441 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a2eda-62c8-443b-9ad0-1cac0bfb977c admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.698439 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1426/2861] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:06:55 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 15:06:55.703385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f263fe-0199-4f0d-ab9a-fe9306f891a5 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.714251 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f263fe-0199-4f0d-ab9a-fe9306f891a5 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.719647 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3677f6-2c2c-45d1-80a7-91dd47609701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.720349 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3677f6-2c2c-45d1-80a7-91dd47609701 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.720586 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3677f6-2c2c-45d1-80a7-91dd47609701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.720906 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3677f6-2c2c-45d1-80a7-91dd47609701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.721597 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3677f6-2c2c-45d1-80a7-91dd47609701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.723745 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3677f6-2c2c-45d1-80a7-91dd47609701 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.735679 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1436/2862] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:55.740258 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a2eda-62c8-443b-9ad0-1cac0bfb977c admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.751066 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a2eda-62c8-443b-9ad0-1cac0bfb977c admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.755853 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a98b5d-447c-409a-b5aa-19dba56689d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.756520 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a98b5d-447c-409a-b5aa-19dba56689d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.756798 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a98b5d-447c-409a-b5aa-19dba56689d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.757071 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a98b5d-447c-409a-b5aa-19dba56689d7 admin admin] PATH_INFO: `/v3/projects/c90f9b8b61cf42cf9e7f40429ce081ae/users/a8770d59b8ed464989dfb6d76f1f9590/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.773161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a98b5d-447c-409a-b5aa-19dba56689d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c90f9b8b61cf42cf9e7f40429ce081ae, user_id=a8770d59b8ed464989dfb6d76f1f9590, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.775871 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a98b5d-447c-409a-b5aa-19dba56689d7 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.789295 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1427/2863] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:55 2022] PUT /identity/v3/projects/c90f9b8b61cf42cf9e7f40429ce081ae/users/a8770d59b8ed464989dfb6d76f1f9590/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:55.793703 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3677f6-2c2c-45d1-80a7-91dd47609701 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.804654 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3677f6-2c2c-45d1-80a7-91dd47609701 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.868137 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1463fcb-869f-4579-82f5-122ca0e3d1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.868838 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1463fcb-869f-4579-82f5-122ca0e3d1b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.869122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1463fcb-869f-4579-82f5-122ca0e3d1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.869396 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1463fcb-869f-4579-82f5-122ca0e3d1b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.870038 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1463fcb-869f-4579-82f5-122ca0e3d1b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.871657 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1463fcb-869f-4579-82f5-122ca0e3d1b2 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.881457 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1437/2864] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:06:55.885896 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a98b5d-447c-409a-b5aa-19dba56689d7 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:55.896866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a98b5d-447c-409a-b5aa-19dba56689d7 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:55.901558 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdfde7-0bcb-4da9-a5ee-cc8a044483f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:55.902208 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdfde7-0bcb-4da9-a5ee-cc8a044483f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.902486 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdfde7-0bcb-4da9-a5ee-cc8a044483f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.902785 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdfde7-0bcb-4da9-a5ee-cc8a044483f0 admin admin] PATH_INFO: `/v3/projects/c90f9b8b61cf42cf9e7f40429ce081ae/users/a8770d59b8ed464989dfb6d76f1f9590/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.904694 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bdfde7-0bcb-4da9-a5ee-cc8a044483f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c90f9b8b61cf42cf9e7f40429ce081ae, user_id=a8770d59b8ed464989dfb6d76f1f9590, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:55.907994 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bdfde7-0bcb-4da9-a5ee-cc8a044483f0 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:55.918208 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1428/2865] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:06:55 2022] PUT /identity/v3/projects/c90f9b8b61cf42cf9e7f40429ce081ae/users/a8770d59b8ed464989dfb6d76f1f9590/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:06:55.929573 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dc3f2-e77d-4114-bbf9-5a01a4fa52b3 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:55.929860 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dc3f2-e77d-4114-bbf9-5a01a4fa52b3 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:55.930134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dc3f2-e77d-4114-bbf9-5a01a4fa52b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:55.958449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-1b6dc3f2-e77d-4114-bbf9-5a01a4fa52b3 None None] MFA Rules not processed for user `a8770d59b8ed464989dfb6d76f1f9590`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:06:55.959933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1b6dc3f2-e77d-4114-bbf9-5a01a4fa52b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:55.971876 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:06:56.028145 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1438/2866] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:06:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:06:56.039373 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdfde7-0bcb-4da9-a5ee-cc8a044483f0 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:06:56.041511 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-85bdfde7-0bcb-4da9-a5ee-cc8a044483f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:06:56.105705 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdfde7-0bcb-4da9-a5ee-cc8a044483f0 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:06:56.109435 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09914649-d0c0-4992-88e6-2937827fdeee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:06:56.110051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09914649-d0c0-4992-88e6-2937827fdeee service cinder] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:06:56.110461 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09914649-d0c0-4992-88e6-2937827fdeee service cinder] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:06:56.110581 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09914649-d0c0-4992-88e6-2937827fdeee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:06:56.167892 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09914649-d0c0-4992-88e6-2937827fdeee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:06:56.169358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09914649-d0c0-4992-88e6-2937827fdeee service cinder] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:06:56.178177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1429/2867] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:06:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:07:01.472257 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6dc3f2-e77d-4114-bbf9-5a01a4fa52b3 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:01.473678 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-1b6dc3f2-e77d-4114-bbf9-5a01a4fa52b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:01.486340 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6dc3f2-e77d-4114-bbf9-5a01a4fa52b3 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:01.489847 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39cc78b-c6ab-47ff-a7c6-6517e2a6e3bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:01.490466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cc78b-c6ab-47ff-a7c6-6517e2a6e3bf service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:01.490755 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cc78b-c6ab-47ff-a7c6-6517e2a6e3bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:01.491049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cc78b-c6ab-47ff-a7c6-6517e2a6e3bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:01.499822 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39cc78b-c6ab-47ff-a7c6-6517e2a6e3bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:01.501202 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39cc78b-c6ab-47ff-a7c6-6517e2a6e3bf service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:01.510618 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1439/2868] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:07:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:07:03.106261 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09914649-d0c0-4992-88e6-2937827fdeee service cinder] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:03.108197 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-09914649-d0c0-4992-88e6-2937827fdeee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:03.145452 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:03.176682 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09914649-d0c0-4992-88e6-2937827fdeee service cinder] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:03.181449 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3500b9-f97a-4a8d-9948-343ef0bb7c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:03.183762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3500b9-f97a-4a8d-9948-343ef0bb7c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:03.183762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3500b9-f97a-4a8d-9948-343ef0bb7c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:03.183762 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3500b9-f97a-4a8d-9948-343ef0bb7c72 admin admin] PATH_INFO: `/v3/users/0c8ee67696ec4441b29c6421c12e1276` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:03.184103 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3500b9-f97a-4a8d-9948-343ef0bb7c72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c8ee67696ec4441b29c6421c12e1276)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:03.185867 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3500b9-f97a-4a8d-9948-343ef0bb7c72 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:03.234638 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3f3500b9-f97a-4a8d-9948-343ef0bb7c72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c8ee67696ec4441b29c6421c12e1276'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:03.242297 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3f3500b9-f97a-4a8d-9948-343ef0bb7c72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c8ee67696ec4441b29c6421c12e1276'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:03.242966 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3f3500b9-f97a-4a8d-9948-343ef0bb7c72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c8ee67696ec4441b29c6421c12e1276'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:03.255298 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3f3500b9-f97a-4a8d-9948-343ef0bb7c72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c8ee67696ec4441b29c6421c12e1276'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:03.256021 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-3f3500b9-f97a-4a8d-9948-343ef0bb7c72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c8ee67696ec4441b29c6421c12e1276'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:03.271870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1430/2869] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:07:03 2022] DELETE /identity/v3/users/0c8ee67696ec4441b29c6421c12e1276 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:03.276971 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41950dec-718f-4264-ac3d-c30478def253 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:03.277776 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41950dec-718f-4264-ac3d-c30478def253 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:03.278076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41950dec-718f-4264-ac3d-c30478def253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:03.297559 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-41950dec-718f-4264-ac3d-c30478def253 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:07:03.298787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-41950dec-718f-4264-ac3d-c30478def253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:03.364426 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1440/2870] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:07:03.374104 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3500b9-f97a-4a8d-9948-343ef0bb7c72 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:03.375570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3f3500b9-f97a-4a8d-9948-343ef0bb7c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:03.433898 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3500b9-f97a-4a8d-9948-343ef0bb7c72 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:03.438073 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f87a7-feea-485a-94ac-a42d6fbe01bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:03.438674 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f87a7-feea-485a-94ac-a42d6fbe01bf service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:03.438983 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f87a7-feea-485a-94ac-a42d6fbe01bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:03.439255 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f87a7-feea-485a-94ac-a42d6fbe01bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:03.448184 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f87a7-feea-485a-94ac-a42d6fbe01bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:03.449605 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f87a7-feea-485a-94ac-a42d6fbe01bf service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:03.459908 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1431/2871] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:07:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:07:03.740061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41950dec-718f-4264-ac3d-c30478def253 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:03.741546 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-41950dec-718f-4264-ac3d-c30478def253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:03.754115 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41950dec-718f-4264-ac3d-c30478def253 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:03.758818 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf23921-60b7-4118-928b-41629b7d68e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:03.759459 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf23921-60b7-4118-928b-41629b7d68e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:03.759759 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf23921-60b7-4118-928b-41629b7d68e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:03.760015 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf23921-60b7-4118-928b-41629b7d68e7 admin admin] PATH_INFO: `/v3/projects/6c7980f54e2645dfbb561a9d8313e234` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:03.761162 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf23921-60b7-4118-928b-41629b7d68e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c7980f54e2645dfbb561a9d8313e234)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:03.763114 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf23921-60b7-4118-928b-41629b7d68e7 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:03.811908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6cf23921-60b7-4118-928b-41629b7d68e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c7980f54e2645dfbb561a9d8313e234', 'request_id': 'req-6cf23921-60b7-4118-928b-41629b7d68e7'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:03.818600 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6cf23921-60b7-4118-928b-41629b7d68e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c7980f54e2645dfbb561a9d8313e234', 'request_id': 'req-6cf23921-60b7-4118-928b-41629b7d68e7'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:03.824212 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6cf23921-60b7-4118-928b-41629b7d68e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c7980f54e2645dfbb561a9d8313e234', 'request_id': 'req-6cf23921-60b7-4118-928b-41629b7d68e7'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:03.837083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6cf23921-60b7-4118-928b-41629b7d68e7 admin admin] The token cache is being invalidate because project 6c7980f54e2645dfbb561a9d8313e234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:07:03.837374 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6cf23921-60b7-4118-928b-41629b7d68e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:03.837997 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-6cf23921-60b7-4118-928b-41629b7d68e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:03.839545 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1441/2872] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:07:03 2022] DELETE /identity/v3/projects/6c7980f54e2645dfbb561a9d8313e234 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:06.420252 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f87a7-feea-485a-94ac-a42d6fbe01bf service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:06.421659 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-3c6f87a7-feea-485a-94ac-a42d6fbe01bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:06.435015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f87a7-feea-485a-94ac-a42d6fbe01bf service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:06.484516 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:06.494531 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9e15c-b7a1-4c99-a85b-a0f5932396f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:06.495364 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9e15c-b7a1-4c99-a85b-a0f5932396f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:06.495870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9e15c-b7a1-4c99-a85b-a0f5932396f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:06.496357 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9e15c-b7a1-4c99-a85b-a0f5932396f3 admin admin] PATH_INFO: `/v3/users/9aa7270ef26f415dad1e860723b94731` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:06.497559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9e15c-b7a1-4c99-a85b-a0f5932396f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aa7270ef26f415dad1e860723b94731)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:06.499509 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9e15c-b7a1-4c99-a85b-a0f5932396f3 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:06.545386 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-06b9e15c-b7a1-4c99-a85b-a0f5932396f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aa7270ef26f415dad1e860723b94731'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:06.553528 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-06b9e15c-b7a1-4c99-a85b-a0f5932396f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aa7270ef26f415dad1e860723b94731'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:06.554238 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-06b9e15c-b7a1-4c99-a85b-a0f5932396f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aa7270ef26f415dad1e860723b94731'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:06.573700 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-06b9e15c-b7a1-4c99-a85b-a0f5932396f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aa7270ef26f415dad1e860723b94731'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:06.574641 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-06b9e15c-b7a1-4c99-a85b-a0f5932396f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aa7270ef26f415dad1e860723b94731'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:06.587791 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1432/2873] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:07:06 2022] DELETE /identity/v3/users/9aa7270ef26f415dad1e860723b94731 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:06.592347 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfc66c-f4c0-49a7-a07a-b54e906bf339 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:06.592844 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfc66c-f4c0-49a7-a07a-b54e906bf339 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:06.593026 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfc66c-f4c0-49a7-a07a-b54e906bf339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:06.613513 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-fedfc66c-f4c0-49a7-a07a-b54e906bf339 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:07:06.614764 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-fedfc66c-f4c0-49a7-a07a-b54e906bf339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:06.630960 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:06.680082 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1442/2874] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:07:06.690080 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9e15c-b7a1-4c99-a85b-a0f5932396f3 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:06.691514 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-06b9e15c-b7a1-4c99-a85b-a0f5932396f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:06.749604 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9e15c-b7a1-4c99-a85b-a0f5932396f3 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:06.753309 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45a32f-1c73-47f0-9cee-a0268ae53ae6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:06.753925 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a32f-1c73-47f0-9cee-a0268ae53ae6 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:06.754185 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a32f-1c73-47f0-9cee-a0268ae53ae6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:06.754455 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a32f-1c73-47f0-9cee-a0268ae53ae6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:06.763542 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45a32f-1c73-47f0-9cee-a0268ae53ae6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:06.764959 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45a32f-1c73-47f0-9cee-a0268ae53ae6 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:06.776258 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1433/2875] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:07:07.044121 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfc66c-f4c0-49a7-a07a-b54e906bf339 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:07.045587 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-fedfc66c-f4c0-49a7-a07a-b54e906bf339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:07.057014 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfc66c-f4c0-49a7-a07a-b54e906bf339 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:07.062145 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1bc1c-cdd3-4116-aaba-29a57909c646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:07.062885 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1bc1c-cdd3-4116-aaba-29a57909c646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:07.063164 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1bc1c-cdd3-4116-aaba-29a57909c646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:07.063424 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff1bc1c-cdd3-4116-aaba-29a57909c646 admin admin] PATH_INFO: `/v3/projects/6d8adb54c72c46c9a05e1668079d4d68` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:07.064594 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1bc1c-cdd3-4116-aaba-29a57909c646 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d8adb54c72c46c9a05e1668079d4d68)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:07.066484 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff1bc1c-cdd3-4116-aaba-29a57909c646 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:07.120170 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-2ff1bc1c-cdd3-4116-aaba-29a57909c646 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d8adb54c72c46c9a05e1668079d4d68', 'request_id': 'req-2ff1bc1c-cdd3-4116-aaba-29a57909c646'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:07.126688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-2ff1bc1c-cdd3-4116-aaba-29a57909c646 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d8adb54c72c46c9a05e1668079d4d68', 'request_id': 'req-2ff1bc1c-cdd3-4116-aaba-29a57909c646'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:07.132318 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-2ff1bc1c-cdd3-4116-aaba-29a57909c646 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d8adb54c72c46c9a05e1668079d4d68', 'request_id': 'req-2ff1bc1c-cdd3-4116-aaba-29a57909c646'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:07.145247 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-2ff1bc1c-cdd3-4116-aaba-29a57909c646 admin admin] The token cache is being invalidate because project 6d8adb54c72c46c9a05e1668079d4d68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:07:07.145533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-2ff1bc1c-cdd3-4116-aaba-29a57909c646 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:07.146173 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-2ff1bc1c-cdd3-4116-aaba-29a57909c646 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:07.147688 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1443/2876] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:07:07 2022] DELETE /identity/v3/projects/6d8adb54c72c46c9a05e1668079d4d68 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:07.151801 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45a32f-1c73-47f0-9cee-a0268ae53ae6 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:07.161349 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45a32f-1c73-47f0-9cee-a0268ae53ae6 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:07.217527 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2100a-0588-4c65-ae85-b5e04d61f995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:07.218149 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2100a-0588-4c65-ae85-b5e04d61f995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:07.218420 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2100a-0588-4c65-ae85-b5e04d61f995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:07.218688 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2100a-0588-4c65-ae85-b5e04d61f995 admin admin] PATH_INFO: `/v3/users/a8770d59b8ed464989dfb6d76f1f9590` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:07.219827 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2100a-0588-4c65-ae85-b5e04d61f995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8770d59b8ed464989dfb6d76f1f9590)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:07.221720 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2100a-0588-4c65-ae85-b5e04d61f995 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:07.242265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:07.270488 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-8fe2100a-0588-4c65-ae85-b5e04d61f995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8770d59b8ed464989dfb6d76f1f9590'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:07.277910 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-8fe2100a-0588-4c65-ae85-b5e04d61f995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8770d59b8ed464989dfb6d76f1f9590'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:07.278610 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-8fe2100a-0588-4c65-ae85-b5e04d61f995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8770d59b8ed464989dfb6d76f1f9590'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:07.290452 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-8fe2100a-0588-4c65-ae85-b5e04d61f995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8770d59b8ed464989dfb6d76f1f9590'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:07.291106 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-8fe2100a-0588-4c65-ae85-b5e04d61f995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8770d59b8ed464989dfb6d76f1f9590'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:07.303133 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1434/2877] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:07:07 2022] DELETE /identity/v3/users/a8770d59b8ed464989dfb6d76f1f9590 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:07.579384 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1bc1c-cdd3-4116-aaba-29a57909c646 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:07.580967 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-2ff1bc1c-cdd3-4116-aaba-29a57909c646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:07.595143 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff1bc1c-cdd3-4116-aaba-29a57909c646 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:07.652015 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d479f-9430-40a7-aa4c-e4272e2f3e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:07.652572 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d479f-9430-40a7-aa4c-e4272e2f3e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:07.652849 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d479f-9430-40a7-aa4c-e4272e2f3e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:07.653120 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d479f-9430-40a7-aa4c-e4272e2f3e80 admin admin] PATH_INFO: `/v3/projects/c90f9b8b61cf42cf9e7f40429ce081ae` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:07.654225 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d479f-9430-40a7-aa4c-e4272e2f3e80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c90f9b8b61cf42cf9e7f40429ce081ae)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:07.656154 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d479f-9430-40a7-aa4c-e4272e2f3e80 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:07.699697 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:07.704029 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f80d479f-9430-40a7-aa4c-e4272e2f3e80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c90f9b8b61cf42cf9e7f40429ce081ae', 'request_id': 'req-f80d479f-9430-40a7-aa4c-e4272e2f3e80'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:07.711235 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f80d479f-9430-40a7-aa4c-e4272e2f3e80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c90f9b8b61cf42cf9e7f40429ce081ae', 'request_id': 'req-f80d479f-9430-40a7-aa4c-e4272e2f3e80'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:07.717082 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f80d479f-9430-40a7-aa4c-e4272e2f3e80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c90f9b8b61cf42cf9e7f40429ce081ae', 'request_id': 'req-f80d479f-9430-40a7-aa4c-e4272e2f3e80'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:07.730911 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f80d479f-9430-40a7-aa4c-e4272e2f3e80 admin admin] The token cache is being invalidate because project c90f9b8b61cf42cf9e7f40429ce081ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:07:07.731199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f80d479f-9430-40a7-aa4c-e4272e2f3e80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:07.731837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f80d479f-9430-40a7-aa4c-e4272e2f3e80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:07.733308 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1444/2878] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:07:07 2022] DELETE /identity/v3/projects/c90f9b8b61cf42cf9e7f40429ce081ae => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:17.299715 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:DELETE=8,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:17.729360 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:26.566512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17449265-1c7a-4f51-8ab4-9ad4e3d4c044 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:26.567538 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17449265-1c7a-4f51-8ab4-9ad4e3d4c044 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:26.567538 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17449265-1c7a-4f51-8ab4-9ad4e3d4c044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:26.599579 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-17449265-1c7a-4f51-8ab4-9ad4e3d4c044 None None] MFA Rules not processed for user `6256407d9df648d5bbe33495cb9a8b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:07:26.600695 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-17449265-1c7a-4f51-8ab4-9ad4e3d4c044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:26.661480 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1435/2879] 10.222.0.26 () {58 vars in 967 bytes} [Thu Sep 22 15:07:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:07:26.701779 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d479f-9430-40a7-aa4c-e4272e2f3e80 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:26.703215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f80d479f-9430-40a7-aa4c-e4272e2f3e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:26.772553 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d479f-9430-40a7-aa4c-e4272e2f3e80 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:26.777486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df72bef-2e53-445d-83bc-434eaa27c5ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a265623ae734936a340c97df852e227', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:26.778095 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df72bef-2e53-445d-83bc-434eaa27c5ee service swift] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:26.778395 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df72bef-2e53-445d-83bc-434eaa27c5ee service swift] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:26.778665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df72bef-2e53-445d-83bc-434eaa27c5ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:26.787271 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df72bef-2e53-445d-83bc-434eaa27c5ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:26.788870 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df72bef-2e53-445d-83bc-434eaa27c5ee service swift] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:26.796905 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1445/2880] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Sep 22 15:07:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 15:07:33.420570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17449265-1c7a-4f51-8ab4-9ad4e3d4c044 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:33.422065 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-17449265-1c7a-4f51-8ab4-9ad4e3d4c044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:33.489623 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17449265-1c7a-4f51-8ab4-9ad4e3d4c044 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:33.494044 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd60cd4-8d53-44ec-978f-dc9e5234a463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:33.494664 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd60cd4-8d53-44ec-978f-dc9e5234a463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:33.495094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd60cd4-8d53-44ec-978f-dc9e5234a463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:33.495368 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd60cd4-8d53-44ec-978f-dc9e5234a463 admin admin] PATH_INFO: `/v3/users/59c1a8f322724f6490fe244c2704e1dd` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:33.496510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd60cd4-8d53-44ec-978f-dc9e5234a463 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59c1a8f322724f6490fe244c2704e1dd)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:33.498535 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd60cd4-8d53-44ec-978f-dc9e5234a463 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:33.544230 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5dd60cd4-8d53-44ec-978f-dc9e5234a463 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59c1a8f322724f6490fe244c2704e1dd'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:33.552269 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5dd60cd4-8d53-44ec-978f-dc9e5234a463 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59c1a8f322724f6490fe244c2704e1dd'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:33.552895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5dd60cd4-8d53-44ec-978f-dc9e5234a463 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59c1a8f322724f6490fe244c2704e1dd'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:33.565309 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5dd60cd4-8d53-44ec-978f-dc9e5234a463 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59c1a8f322724f6490fe244c2704e1dd'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:33.565963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-5dd60cd4-8d53-44ec-978f-dc9e5234a463 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59c1a8f322724f6490fe244c2704e1dd'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:33.578316 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1436/2881] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:07:33 2022] DELETE /identity/v3/users/59c1a8f322724f6490fe244c2704e1dd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:33.754083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df72bef-2e53-445d-83bc-434eaa27c5ee service swift] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:33.769069 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df72bef-2e53-445d-83bc-434eaa27c5ee service swift] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:33.825815 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4729130-7648-4cb0-88db-2e04788c42f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:33.826435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4729130-7648-4cb0-88db-2e04788c42f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:33.826710 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4729130-7648-4cb0-88db-2e04788c42f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:33.826998 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4729130-7648-4cb0-88db-2e04788c42f8 admin admin] PATH_INFO: `/v3/projects/3a021020e4254a54b5892a11537a4807` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:33.828115 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4729130-7648-4cb0-88db-2e04788c42f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a021020e4254a54b5892a11537a4807)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:33.830504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4729130-7648-4cb0-88db-2e04788c42f8 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:33.877958 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f4729130-7648-4cb0-88db-2e04788c42f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a021020e4254a54b5892a11537a4807', 'request_id': 'req-f4729130-7648-4cb0-88db-2e04788c42f8'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:33.884061 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f4729130-7648-4cb0-88db-2e04788c42f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a021020e4254a54b5892a11537a4807', 'request_id': 'req-f4729130-7648-4cb0-88db-2e04788c42f8'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:33.889554 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f4729130-7648-4cb0-88db-2e04788c42f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a021020e4254a54b5892a11537a4807', 'request_id': 'req-f4729130-7648-4cb0-88db-2e04788c42f8'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:33.902390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f4729130-7648-4cb0-88db-2e04788c42f8 admin admin] The token cache is being invalidate because project 3a021020e4254a54b5892a11537a4807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:07:33.902663 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f4729130-7648-4cb0-88db-2e04788c42f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:33.903307 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-f4729130-7648-4cb0-88db-2e04788c42f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:33.904773 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1446/2882] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:07:33 2022] DELETE /identity/v3/projects/3a021020e4254a54b5892a11537a4807 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:33.909427 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd60cd4-8d53-44ec-978f-dc9e5234a463 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:33.910911 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-5dd60cd4-8d53-44ec-978f-dc9e5234a463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:33.923732 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd60cd4-8d53-44ec-978f-dc9e5234a463 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:33.928268 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:33.981897 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35db28-0f6b-42ff-a2e1-b213ffb945c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:33.982510 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35db28-0f6b-42ff-a2e1-b213ffb945c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:33.982817 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35db28-0f6b-42ff-a2e1-b213ffb945c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:33.983087 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35db28-0f6b-42ff-a2e1-b213ffb945c5 admin admin] PATH_INFO: `/v3/users/c75e234177fb4ed1a759e609b20da2c4` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:33.984202 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35db28-0f6b-42ff-a2e1-b213ffb945c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c75e234177fb4ed1a759e609b20da2c4)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:33.986093 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35db28-0f6b-42ff-a2e1-b213ffb945c5 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:34.034062 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0b35db28-0f6b-42ff-a2e1-b213ffb945c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c75e234177fb4ed1a759e609b20da2c4'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:34.041358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0b35db28-0f6b-42ff-a2e1-b213ffb945c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c75e234177fb4ed1a759e609b20da2c4'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:34.042159 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0b35db28-0f6b-42ff-a2e1-b213ffb945c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c75e234177fb4ed1a759e609b20da2c4'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:34.054189 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0b35db28-0f6b-42ff-a2e1-b213ffb945c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c75e234177fb4ed1a759e609b20da2c4'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:34.054874 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-0b35db28-0f6b-42ff-a2e1-b213ffb945c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c75e234177fb4ed1a759e609b20da2c4'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:34.067075 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1437/2883] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:07:33 2022] DELETE /identity/v3/users/c75e234177fb4ed1a759e609b20da2c4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:07:34.247100 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4729130-7648-4cb0-88db-2e04788c42f8 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:07:34.248726 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f4729130-7648-4cb0-88db-2e04788c42f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:07:34.265656 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4729130-7648-4cb0-88db-2e04788c42f8 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:07:34.284390 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:34.326035 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:07:34.326677 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:07:34.327029 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:07:34.327307 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0 admin admin] PATH_INFO: `/v3/projects/91f9212574384cac856b83b0e813f747` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:07:34.328437 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f9212574384cac856b83b0e813f747)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:07:34.330963 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:07:34.381211 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f9212574384cac856b83b0e813f747', 'request_id': 'req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:34.389027 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f9212574384cac856b83b0e813f747', 'request_id': 'req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:34.397308 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f9212574384cac856b83b0e813f747', 'request_id': 'req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:34.411140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0 admin admin] The token cache is being invalidate because project 91f9212574384cac856b83b0e813f747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:07:34.411428 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:34.412049 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:07:34.413563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1447/2884] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:07:34 2022] DELETE /identity/v3/projects/91f9212574384cac856b83b0e813f747 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:07:44.063484 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:07:44.408520 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:08:25.041604 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35db28-0f6b-42ff-a2e1-b213ffb945c5 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:08:25.043107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-0b35db28-0f6b-42ff-a2e1-b213ffb945c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:25.100821 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35db28-0f6b-42ff-a2e1-b213ffb945c5 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:08:25.151086 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eeae8-9055-48a9-b3b1-61d263547397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:08:25.151695 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eeae8-9055-48a9-b3b1-61d263547397 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:25.151973 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eeae8-9055-48a9-b3b1-61d263547397 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:25.152244 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eeae8-9055-48a9-b3b1-61d263547397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:25.153965 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-1e0eeae8-9055-48a9-b3b1-61d263547397 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:25.237237 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0eeae8-9055-48a9-b3b1-61d263547397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:08:25.238718 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0eeae8-9055-48a9-b3b1-61d263547397 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:08:25.259030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1438/2885] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:08:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:08:26.446629 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:08:26.448078 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:26.461875 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba22f9-2bac-407f-b009-3f3dc5ba85e0 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:08:26.466033 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c976ee-ad44-42c0-bcb1-5666e48b8b13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:08:26.466641 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c976ee-ad44-42c0-bcb1-5666e48b8b13 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:26.467037 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c976ee-ad44-42c0-bcb1-5666e48b8b13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:26.467340 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c976ee-ad44-42c0-bcb1-5666e48b8b13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:26.469099 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a4c976ee-ad44-42c0-bcb1-5666e48b8b13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:26.484315 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c976ee-ad44-42c0-bcb1-5666e48b8b13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:08:26.485744 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c976ee-ad44-42c0-bcb1-5666e48b8b13 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:08:26.498131 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1448/2886] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:08:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:08:35.256492 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=83 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:08:36.492908 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=6 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:08:38.431095 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eeae8-9055-48a9-b3b1-61d263547397 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:08:38.439686 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eeae8-9055-48a9-b3b1-61d263547397 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:08:38.443330 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecc153-9e29-48cf-bbce-b2683e494352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:08:38.443925 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecc153-9e29-48cf-bbce-b2683e494352 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:38.444201 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecc153-9e29-48cf-bbce-b2683e494352 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:38.444473 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecc153-9e29-48cf-bbce-b2683e494352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:38.446256 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-84ecc153-9e29-48cf-bbce-b2683e494352 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:38.455931 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ecc153-9e29-48cf-bbce-b2683e494352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:08:38.457678 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ecc153-9e29-48cf-bbce-b2683e494352 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:08:38.466800 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1439/2887] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:08:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:08:39.600462 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3bbef-de48-4ca4-86fb-69fa6875fec4 None None] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:39.601084 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3bbef-de48-4ca4-86fb-69fa6875fec4 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:39.601084 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a3bbef-de48-4ca4-86fb-69fa6875fec4 None None] PATH_INFO: `/` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:39.602371 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1449/2888] 10.222.0.26 () {54 vars in 921 bytes} [Thu Sep 22 15:08:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 15:08:39.608505 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86a4ea-6468-4ce9-811b-1e8f3a576f9c None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:39.609011 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86a4ea-6468-4ce9-811b-1e8f3a576f9c None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:39.609120 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86a4ea-6468-4ce9-811b-1e8f3a576f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:39.647413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-aa86a4ea-6468-4ce9-811b-1e8f3a576f9c None None] MFA Rules not processed for user `f77bbd8de87042aab3674db366635cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:08:39.648649 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-aa86a4ea-6468-4ce9-811b-1e8f3a576f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:39.655506 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1440/2889] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Sep 22 15:08:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:08:39.669606 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a3bbef-de48-4ca4-86fb-69fa6875fec4 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:08:39.671421 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-15a3bbef-de48-4ca4-86fb-69fa6875fec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:39.686011 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a3bbef-de48-4ca4-86fb-69fa6875fec4 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:08:39.689765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc65ed-d0f6-4186-a6b7-054929f8aea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:08:39.690435 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc65ed-d0f6-4186-a6b7-054929f8aea5 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:39.690811 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc65ed-d0f6-4186-a6b7-054929f8aea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:39.691152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc65ed-d0f6-4186-a6b7-054929f8aea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:39.698951 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc65ed-d0f6-4186-a6b7-054929f8aea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:08:39.700385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc65ed-d0f6-4186-a6b7-054929f8aea5 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:08:39.709199 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1450/2890] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:08:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:08:39.796828 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7566783-2dd7-4f80-9a7b-beea3762bbe3 None None] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:39.797139 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7566783-2dd7-4f80-9a7b-beea3762bbe3 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:39.797412 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7566783-2dd7-4f80-9a7b-beea3762bbe3 None None] PATH_INFO: `/` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:39.798899 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1441/2891] 10.222.0.26 () {54 vars in 921 bytes} [Thu Sep 22 15:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 15:08:39.804173 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b25b3-6836-4a7f-b136-9a283f742fc2 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:39.804460 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b25b3-6836-4a7f-b136-9a283f742fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:39.804737 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b25b3-6836-4a7f-b136-9a283f742fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:39.842803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-8c8b25b3-6836-4a7f-b136-9a283f742fc2 None None] MFA Rules not processed for user `0c9be41c39494e54a14398a1f03c4d69`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:08:39.843946 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-8c8b25b3-6836-4a7f-b136-9a283f742fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:39.853904 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1451/2892] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Sep 22 15:08:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:08:39.864895 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7566783-2dd7-4f80-9a7b-beea3762bbe3 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:08:39.876010 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7566783-2dd7-4f80-9a7b-beea3762bbe3 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:08:39.879512 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a37ad-fece-4c56-b8ed-45589793fae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:08:39.880107 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a37ad-fece-4c56-b8ed-45589793fae1 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:39.880385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a37ad-fece-4c56-b8ed-45589793fae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:39.880657 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a37ad-fece-4c56-b8ed-45589793fae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:39.890614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a37ad-fece-4c56-b8ed-45589793fae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:08:39.892485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a37ad-fece-4c56-b8ed-45589793fae1 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:08:39.904182 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1442/2893] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:08:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:08:40.222403 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b25b3-6836-4a7f-b136-9a283f742fc2 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:08:40.223802 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-8c8b25b3-6836-4a7f-b136-9a283f742fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:40.295257 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b25b3-6836-4a7f-b136-9a283f742fc2 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:08:40.300204 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6855f-b24d-4373-9403-c7c19d68733a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b6b34b4ee5492fb1f0699716e5118b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:08:40.300800 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6855f-b24d-4373-9403-c7c19d68733a service placement] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:40.301076 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6855f-b24d-4373-9403-c7c19d68733a service placement] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:40.301345 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6855f-b24d-4373-9403-c7c19d68733a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:40.303081 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a2d6855f-b24d-4373-9403-c7c19d68733a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:40.315262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6855f-b24d-4373-9403-c7c19d68733a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:08:40.316829 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6855f-b24d-4373-9403-c7c19d68733a service placement] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:08:40.327464 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1452/2894] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:08:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:08:41.575942 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a37ad-fece-4c56-b8ed-45589793fae1 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:08:41.585430 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a37ad-fece-4c56-b8ed-45589793fae1 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:08:41.589313 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959e94f-dfc1-410c-9270-11617ec6c037 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:08:41.589918 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959e94f-dfc1-410c-9270-11617ec6c037 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:08:41.590201 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959e94f-dfc1-410c-9270-11617ec6c037 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:08:41.590476 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959e94f-dfc1-410c-9270-11617ec6c037 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:08:41.592277 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-7959e94f-dfc1-410c-9270-11617ec6c037 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:08:41.602870 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7959e94f-dfc1-410c-9270-11617ec6c037 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:08:41.604338 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7959e94f-dfc1-410c-9270-11617ec6c037 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:08:41.613406 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1443/2895] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:08:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:08:50.323134 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=47 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:08:51.611109 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=27 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:10:06.815407 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6855f-b24d-4373-9403-c7c19d68733a service placement] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:06.816886 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a2d6855f-b24d-4373-9403-c7c19d68733a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:06.836306 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6855f-b24d-4373-9403-c7c19d68733a service placement] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:06.841824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b21a4-0b4d-49c6-9eb3-ddd093ca3cae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:06.842683 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b21a4-0b4d-49c6-9eb3-ddd093ca3cae service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:06.842986 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b21a4-0b4d-49c6-9eb3-ddd093ca3cae service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:06.843255 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b21a4-0b4d-49c6-9eb3-ddd093ca3cae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:06.845044 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-d60b21a4-0b4d-49c6-9eb3-ddd093ca3cae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:06.865222 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b21a4-0b4d-49c6-9eb3-ddd093ca3cae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:06.865222 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b21a4-0b4d-49c6-9eb3-ddd093ca3cae service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:06.878207 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1453/2896] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:10:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:10:13.714529 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959e94f-dfc1-410c-9270-11617ec6c037 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:13.732154 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959e94f-dfc1-410c-9270-11617ec6c037 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:13.739751 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0ac82-d9f2-4093-ada9-e817a023e7d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b6b34b4ee5492fb1f0699716e5118b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:13.740614 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0ac82-d9f2-4093-ada9-e817a023e7d3 service placement] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:13.741055 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0ac82-d9f2-4093-ada9-e817a023e7d3 service placement] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:13.741513 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0ac82-d9f2-4093-ada9-e817a023e7d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:13.743401 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-41e0ac82-d9f2-4093-ada9-e817a023e7d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:13.756536 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0ac82-d9f2-4093-ada9-e817a023e7d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:13.758162 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0ac82-d9f2-4093-ada9-e817a023e7d3 service placement] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:13.771475 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1444/2897] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:10:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:10:16.864103 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b21a4-0b4d-49c6-9eb3-ddd093ca3cae service nova] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:16.878138 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b21a4-0b4d-49c6-9eb3-ddd093ca3cae service nova] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:16.883665 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828fc8e-64db-4cb1-a368-dd9809b1a8fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b6b34b4ee5492fb1f0699716e5118b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:16.884261 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828fc8e-64db-4cb1-a368-dd9809b1a8fa service placement] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:16.884533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828fc8e-64db-4cb1-a368-dd9809b1a8fa service placement] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:16.884797 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828fc8e-64db-4cb1-a368-dd9809b1a8fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:16.886506 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-9828fc8e-64db-4cb1-a368-dd9809b1a8fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:16.899664 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828fc8e-64db-4cb1-a368-dd9809b1a8fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:16.901462 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828fc8e-64db-4cb1-a368-dd9809b1a8fa service placement] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:16.917086 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1454/2898] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:10:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:10:16.930830 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0ac82-d9f2-4093-ada9-e817a023e7d3 service placement] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:16.932368 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-41e0ac82-d9f2-4093-ada9-e817a023e7d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:16.998483 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0ac82-d9f2-4093-ada9-e817a023e7d3 service placement] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:17.003169 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfc631-ca01-44a0-9b10-e2d182de4eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:17.003797 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfc631-ca01-44a0-9b10-e2d182de4eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:17.004078 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfc631-ca01-44a0-9b10-e2d182de4eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:17.004354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfc631-ca01-44a0-9b10-e2d182de4eb9 admin admin] PATH_INFO: `/v3/users/b761e143a6264e66aac0caac8cb46135` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:17.006012 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfc631-ca01-44a0-9b10-e2d182de4eb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b761e143a6264e66aac0caac8cb46135)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:17.007958 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfc631-ca01-44a0-9b10-e2d182de4eb9 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:17.053866 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-1acfc631-ca01-44a0-9b10-e2d182de4eb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b761e143a6264e66aac0caac8cb46135'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.061017 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-1acfc631-ca01-44a0-9b10-e2d182de4eb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b761e143a6264e66aac0caac8cb46135'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.061640 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-1acfc631-ca01-44a0-9b10-e2d182de4eb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b761e143a6264e66aac0caac8cb46135'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.074001 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-1acfc631-ca01-44a0-9b10-e2d182de4eb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b761e143a6264e66aac0caac8cb46135'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.074645 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-1acfc631-ca01-44a0-9b10-e2d182de4eb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b761e143a6264e66aac0caac8cb46135'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.087389 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1445/2899] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:10:16 2022] DELETE /identity/v3/users/b761e143a6264e66aac0caac8cb46135 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:10:17.260866 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828fc8e-64db-4cb1-a368-dd9809b1a8fa service placement] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:17.272005 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828fc8e-64db-4cb1-a368-dd9809b1a8fa service placement] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:17.330184 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:17.330856 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:17.331510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:17.331510 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd admin admin] PATH_INFO: `/v3/projects/ab6ca70d427646d3b0a8ebd2838c807b` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:17.332748 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab6ca70d427646d3b0a8ebd2838c807b)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:17.334857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:17.382441 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab6ca70d427646d3b0a8ebd2838c807b', 'request_id': 'req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.388449 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab6ca70d427646d3b0a8ebd2838c807b', 'request_id': 'req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.394330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab6ca70d427646d3b0a8ebd2838c807b', 'request_id': 'req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.406353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd admin admin] The token cache is being invalidate because project ab6ca70d427646d3b0a8ebd2838c807b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:10:17.406633 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.407267 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.408808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1455/2900] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:10:17 2022] DELETE /identity/v3/projects/ab6ca70d427646d3b0a8ebd2838c807b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:10:17.412838 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfc631-ca01-44a0-9b10-e2d182de4eb9 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:17.414226 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-1acfc631-ca01-44a0-9b10-e2d182de4eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:17.427550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfc631-ca01-44a0-9b10-e2d182de4eb9 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:17.482829 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134bd58c-32ee-443d-b3ef-e127c76cd872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:17.483399 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bd58c-32ee-443d-b3ef-e127c76cd872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:17.483680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bd58c-32ee-443d-b3ef-e127c76cd872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:17.483952 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134bd58c-32ee-443d-b3ef-e127c76cd872 admin admin] PATH_INFO: `/v3/users/834183ae750241d7a1e4c4bc3d50f572` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:17.485091 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134bd58c-32ee-443d-b3ef-e127c76cd872 admin admin] RBAC: Authorizing `identity:delete_user(user_id=834183ae750241d7a1e4c4bc3d50f572)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:17.487028 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134bd58c-32ee-443d-b3ef-e127c76cd872 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:17.492851 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:10:17.534537 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-134bd58c-32ee-443d-b3ef-e127c76cd872 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '834183ae750241d7a1e4c4bc3d50f572'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.542240 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-134bd58c-32ee-443d-b3ef-e127c76cd872 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '834183ae750241d7a1e4c4bc3d50f572'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.542871 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-134bd58c-32ee-443d-b3ef-e127c76cd872 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '834183ae750241d7a1e4c4bc3d50f572'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.556970 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-134bd58c-32ee-443d-b3ef-e127c76cd872 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '834183ae750241d7a1e4c4bc3d50f572'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.557800 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-134bd58c-32ee-443d-b3ef-e127c76cd872 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '834183ae750241d7a1e4c4bc3d50f572'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.571814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1446/2901] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:10:17 2022] DELETE /identity/v3/users/834183ae750241d7a1e4c4bc3d50f572 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:10:17.744803 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:17.746765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:17.760727 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aa444-2e92-4a2b-9cd1-8675f8cf21bd admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:17.822913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc960b6d-436f-460c-8c5f-71c0f3b3f8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:17.823563 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc960b6d-436f-460c-8c5f-71c0f3b3f8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:17.823841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc960b6d-436f-460c-8c5f-71c0f3b3f8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:17.824112 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc960b6d-436f-460c-8c5f-71c0f3b3f8d7 admin admin] PATH_INFO: `/v3/projects/9f145f38cb0b472186ee1ea62ccf702b` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:17.825233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc960b6d-436f-460c-8c5f-71c0f3b3f8d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f145f38cb0b472186ee1ea62ccf702b)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:17.827189 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc960b6d-436f-460c-8c5f-71c0f3b3f8d7 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:17.833700 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:10:17.880233 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-bc960b6d-436f-460c-8c5f-71c0f3b3f8d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f145f38cb0b472186ee1ea62ccf702b', 'request_id': 'req-bc960b6d-436f-460c-8c5f-71c0f3b3f8d7'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.886675 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-bc960b6d-436f-460c-8c5f-71c0f3b3f8d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f145f38cb0b472186ee1ea62ccf702b', 'request_id': 'req-bc960b6d-436f-460c-8c5f-71c0f3b3f8d7'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.893183 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-bc960b6d-436f-460c-8c5f-71c0f3b3f8d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f145f38cb0b472186ee1ea62ccf702b', 'request_id': 'req-bc960b6d-436f-460c-8c5f-71c0f3b3f8d7'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.904928 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-bc960b6d-436f-460c-8c5f-71c0f3b3f8d7 admin admin] The token cache is being invalidate because project 9f145f38cb0b472186ee1ea62ccf702b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81454) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:10:17.905215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-bc960b6d-436f-460c-8c5f-71c0f3b3f8d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.905837 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-bc960b6d-436f-460c-8c5f-71c0f3b3f8d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:10:17.907366 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1456/2902] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:10:17 2022] DELETE /identity/v3/projects/9f145f38cb0b472186ee1ea62ccf702b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:10:17.914177 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff862a82-5818-48af-83b9-fcf1ed3b48c4 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:17.914723 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff862a82-5818-48af-83b9-fcf1ed3b48c4 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:17.915301 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff862a82-5818-48af-83b9-fcf1ed3b48c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:17.938031 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-ff862a82-5818-48af-83b9-fcf1ed3b48c4 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:10:17.939282 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ff862a82-5818-48af-83b9-fcf1ed3b48c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:17.998429 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1447/2903] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:10:18.006365 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a507d3-6302-451d-b73b-ad7a4145d490 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:18.006643 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a507d3-6302-451d-b73b-ad7a4145d490 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:18.006947 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a507d3-6302-451d-b73b-ad7a4145d490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:18.026859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-27a507d3-6302-451d-b73b-ad7a4145d490 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:10:18.028075 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-27a507d3-6302-451d-b73b-ad7a4145d490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:18.035997 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1457/2904] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:10:18.041446 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff862a82-5818-48af-83b9-fcf1ed3b48c4 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:18.051146 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff862a82-5818-48af-83b9-fcf1ed3b48c4 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:18.055992 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbadd8-6165-48d6-a279-b858cf72e079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:18.056647 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbadd8-6165-48d6-a279-b858cf72e079 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:18.056925 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbadd8-6165-48d6-a279-b858cf72e079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:18.057196 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbadd8-6165-48d6-a279-b858cf72e079 admin admin] PATH_INFO: `/v3/domains` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:18.057860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fbadd8-6165-48d6-a279-b858cf72e079 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81455) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 15:10:18.058164 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fbadd8-6165-48d6-a279-b858cf72e079 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:18.059814 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fbadd8-6165-48d6-a279-b858cf72e079 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:18.071520 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1448/2905] 10.222.0.26 () {58 vars in 1153 bytes} [Thu Sep 22 15:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 15:10:18.075859 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a507d3-6302-451d-b73b-ad7a4145d490 None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:18.085473 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a507d3-6302-451d-b73b-ad7a4145d490 None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:18.090145 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd942b46-1bea-4b00-80d3-4ce90626edc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:18.090765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd942b46-1bea-4b00-80d3-4ce90626edc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:18.091046 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd942b46-1bea-4b00-80d3-4ce90626edc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:18.091312 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd942b46-1bea-4b00-80d3-4ce90626edc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:18.091943 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd942b46-1bea-4b00-80d3-4ce90626edc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:18.094073 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd942b46-1bea-4b00-80d3-4ce90626edc0 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:18.132913 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1458/2906] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:10:18 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:10:18.137216 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbadd8-6165-48d6-a279-b858cf72e079 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:18.148788 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbadd8-6165-48d6-a279-b858cf72e079 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:18.204206 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20a37e-48ca-47ff-bfb3-03782e9917a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:18.204810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20a37e-48ca-47ff-bfb3-03782e9917a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:18.205090 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20a37e-48ca-47ff-bfb3-03782e9917a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:18.205363 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20a37e-48ca-47ff-bfb3-03782e9917a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:18.206003 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20a37e-48ca-47ff-bfb3-03782e9917a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:18.207928 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20a37e-48ca-47ff-bfb3-03782e9917a7 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:18.230802 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1449/2907] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:10:18 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:10:18.235072 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd942b46-1bea-4b00-80d3-4ce90626edc0 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:18.245597 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd942b46-1bea-4b00-80d3-4ce90626edc0 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:18.250196 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f007f-62c0-42aa-9cc2-9bc1bfd75d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:18.250725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f007f-62c0-42aa-9cc2-9bc1bfd75d84 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:18.251025 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f007f-62c0-42aa-9cc2-9bc1bfd75d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:18.251291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f007f-62c0-42aa-9cc2-9bc1bfd75d84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:18.252194 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f007f-62c0-42aa-9cc2-9bc1bfd75d84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:18.253765 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f007f-62c0-42aa-9cc2-9bc1bfd75d84 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:18.263653 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1459/2908] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:10:18.267519 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20a37e-48ca-47ff-bfb3-03782e9917a7 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:18.273038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:10:18.279340 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20a37e-48ca-47ff-bfb3-03782e9917a7 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:18.284955 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e83e6b-f188-4ad1-96fa-5126d6f3e644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:18.285572 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e83e6b-f188-4ad1-96fa-5126d6f3e644 admin admin] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:18.285846 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e83e6b-f188-4ad1-96fa-5126d6f3e644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:18.286113 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e83e6b-f188-4ad1-96fa-5126d6f3e644 admin admin] PATH_INFO: `/v3/roles` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:18.287362 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e83e6b-f188-4ad1-96fa-5126d6f3e644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:18.288938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e83e6b-f188-4ad1-96fa-5126d6f3e644 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:18.299495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1450/2909] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:10:18.304093 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f007f-62c0-42aa-9cc2-9bc1bfd75d84 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:18.314654 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f007f-62c0-42aa-9cc2-9bc1bfd75d84 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:18.319845 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378dbe2-16f2-49ce-988b-281da1e45172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:18.320493 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378dbe2-16f2-49ce-988b-281da1e45172 admin admin] REQUEST_METHOD: `PUT` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:18.320769 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378dbe2-16f2-49ce-988b-281da1e45172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:18.321038 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378dbe2-16f2-49ce-988b-281da1e45172 admin admin] PATH_INFO: `/v3/projects/1b64f192d3ea436abc310162b1c46efc/users/8d49d3422c7d420db80fe5814a768f53/roles/7d3cf58c1ad2407c9f8c60ba99b2006d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:18.336661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378dbe2-16f2-49ce-988b-281da1e45172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b64f192d3ea436abc310162b1c46efc, user_id=8d49d3422c7d420db80fe5814a768f53, role_id=7d3cf58c1ad2407c9f8c60ba99b2006d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:18.339577 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378dbe2-16f2-49ce-988b-281da1e45172 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:18.352970 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1460/2910] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:10:18 2022] PUT /identity/v3/projects/1b64f192d3ea436abc310162b1c46efc/users/8d49d3422c7d420db80fe5814a768f53/roles/7d3cf58c1ad2407c9f8c60ba99b2006d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:10:18.358008 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb0e15-572e-491d-8215-7f87ebd2d710 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:18.358290 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb0e15-572e-491d-8215-7f87ebd2d710 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:18.358565 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb0e15-572e-491d-8215-7f87ebd2d710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:18.377639 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-ebdb0e15-572e-491d-8215-7f87ebd2d710 None None] MFA Rules not processed for user `4c56fbc6ee754a96af39128dae8f43a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:10:18.378858 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-ebdb0e15-572e-491d-8215-7f87ebd2d710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:18.437501 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1451/2911] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:10:18.447879 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378dbe2-16f2-49ce-988b-281da1e45172 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:18.449591 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-e378dbe2-16f2-49ce-988b-281da1e45172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:18.505461 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378dbe2-16f2-49ce-988b-281da1e45172 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:18.509009 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab68bc-1681-4db0-ad60-effbd58d3166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:18.509608 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab68bc-1681-4db0-ad60-effbd58d3166 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:18.509884 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab68bc-1681-4db0-ad60-effbd58d3166 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:18.510152 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab68bc-1681-4db0-ad60-effbd58d3166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:18.519079 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab68bc-1681-4db0-ad60-effbd58d3166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:18.520496 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab68bc-1681-4db0-ad60-effbd58d3166 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:18.530276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1461/2912] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:10:22.530427 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d4364-5702-4869-afdf-155958482f0f None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:22.531233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d4364-5702-4869-afdf-155958482f0f None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:22.531233 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d4364-5702-4869-afdf-155958482f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:22.562880 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-4f7d4364-5702-4869-afdf-155958482f0f None None] MFA Rules not processed for user `8d49d3422c7d420db80fe5814a768f53`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:10:22.564124 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-4f7d4364-5702-4869-afdf-155958482f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:22.635844 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1452/2913] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:10:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:10:22.640577 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab68bc-1681-4db0-ad60-effbd58d3166 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:22.651291 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab68bc-1681-4db0-ad60-effbd58d3166 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:22.656011 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f9a17-fd77-442b-9d32-354a686922e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:22.656612 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f9a17-fd77-442b-9d32-354a686922e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:22.656885 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f9a17-fd77-442b-9d32-354a686922e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:22.657149 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f9a17-fd77-442b-9d32-354a686922e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:22.657770 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f9a17-fd77-442b-9d32-354a686922e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:22.659731 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f9a17-fd77-442b-9d32-354a686922e3 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:22.696289 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1462/2914] 10.222.0.26 () {60 vars in 1153 bytes} [Thu Sep 22 15:10:22 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:10:22.700655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d4364-5702-4869-afdf-155958482f0f None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:22.722655 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d4364-5702-4869-afdf-155958482f0f None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:22.788381 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27034a98-4e5d-43f3-8aa9-c2a3818e5f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:22.789089 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27034a98-4e5d-43f3-8aa9-c2a3818e5f14 admin admin] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:22.789464 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27034a98-4e5d-43f3-8aa9-c2a3818e5f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:22.789739 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27034a98-4e5d-43f3-8aa9-c2a3818e5f14 admin admin] PATH_INFO: `/v3/users` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:22.790392 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27034a98-4e5d-43f3-8aa9-c2a3818e5f14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:22.792363 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27034a98-4e5d-43f3-8aa9-c2a3818e5f14 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:22.816015 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1453/2915] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Sep 22 15:10:22 2022] POST /identity/v3/users => generated 295 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 15:10:22.820486 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f9a17-fd77-442b-9d32-354a686922e3 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:22.826232 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:10:22.838561 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f9a17-fd77-442b-9d32-354a686922e3 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:22.844238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ac835-39df-4c26-bb3e-be9fb2d05536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:22.844867 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ac835-39df-4c26-bb3e-be9fb2d05536 admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:22.845223 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ac835-39df-4c26-bb3e-be9fb2d05536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:22.845507 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ac835-39df-4c26-bb3e-be9fb2d05536 admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:22.846157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ac835-39df-4c26-bb3e-be9fb2d05536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:22.847813 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ac835-39df-4c26-bb3e-be9fb2d05536 admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:22.860531 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1463/2916] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:10:22.865414 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27034a98-4e5d-43f3-8aa9-c2a3818e5f14 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:22.877215 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27034a98-4e5d-43f3-8aa9-c2a3818e5f14 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:22.883826 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafaaae-ba7e-4139-adf2-c2c9aaa391a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:22.884314 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafaaae-ba7e-4139-adf2-c2c9aaa391a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:22.884597 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafaaae-ba7e-4139-adf2-c2c9aaa391a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:22.884881 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafaaae-ba7e-4139-adf2-c2c9aaa391a5 admin admin] PATH_INFO: `/v3/projects/8f9e9a78f2fc497787dc8981cc54e770/users/76d2d9b7116e46b0a93b5aabd019018d/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:22.896362 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:10:22.907066 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafaaae-ba7e-4139-adf2-c2c9aaa391a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9e9a78f2fc497787dc8981cc54e770, user_id=76d2d9b7116e46b0a93b5aabd019018d, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:22.910069 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafaaae-ba7e-4139-adf2-c2c9aaa391a5 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:22.935097 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1454/2917] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:10:22 2022] PUT /identity/v3/projects/8f9e9a78f2fc497787dc8981cc54e770/users/76d2d9b7116e46b0a93b5aabd019018d/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:10:22.939322 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ac835-39df-4c26-bb3e-be9fb2d05536 admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:22.952766 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ac835-39df-4c26-bb3e-be9fb2d05536 admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:23.014150 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d5d28b-61c7-4366-bb87-e0b70a2429af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:23.014807 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5d28b-61c7-4366-bb87-e0b70a2429af admin admin] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:23.015077 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5d28b-61c7-4366-bb87-e0b70a2429af admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:23.015346 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d5d28b-61c7-4366-bb87-e0b70a2429af admin admin] PATH_INFO: `/v3/roles` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:23.015986 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5d28b-61c7-4366-bb87-e0b70a2429af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:23.017586 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d5d28b-61c7-4366-bb87-e0b70a2429af admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:23.028256 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1464/2918] 10.222.0.26 () {58 vars in 1122 bytes} [Thu Sep 22 15:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 15:10:23.034251 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafaaae-ba7e-4139-adf2-c2c9aaa391a5 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:23.045419 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafaaae-ba7e-4139-adf2-c2c9aaa391a5 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:23.050223 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1ae51-ee0d-459b-8aaa-d6f45b7ed72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:23.051024 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1ae51-ee0d-459b-8aaa-d6f45b7ed72b admin admin] REQUEST_METHOD: `PUT` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:23.051369 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1ae51-ee0d-459b-8aaa-d6f45b7ed72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:23.051706 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1ae51-ee0d-459b-8aaa-d6f45b7ed72b admin admin] PATH_INFO: `/v3/projects/8f9e9a78f2fc497787dc8981cc54e770/users/76d2d9b7116e46b0a93b5aabd019018d/roles/32c8a81e339a462e973e76bb528ef35a` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:23.054333 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1ae51-ee0d-459b-8aaa-d6f45b7ed72b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9e9a78f2fc497787dc8981cc54e770, user_id=76d2d9b7116e46b0a93b5aabd019018d, role_id=32c8a81e339a462e973e76bb528ef35a)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:23.057045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1ae51-ee0d-459b-8aaa-d6f45b7ed72b admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:23.068479 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1455/2919] 10.222.0.26 () {60 vars in 1483 bytes} [Thu Sep 22 15:10:23 2022] PUT /identity/v3/projects/8f9e9a78f2fc497787dc8981cc54e770/users/76d2d9b7116e46b0a93b5aabd019018d/roles/32c8a81e339a462e973e76bb528ef35a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 15:10:28.051644 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b2e06-52c1-4324-893d-dfe08699a6d8 None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:28.052208 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b2e06-52c1-4324-893d-dfe08699a6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:28.052320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b2e06-52c1-4324-893d-dfe08699a6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:28.086466 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-7f8b2e06-52c1-4324-893d-dfe08699a6d8 None None] MFA Rules not processed for user `76d2d9b7116e46b0a93b5aabd019018d`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:10:28.087774 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-7f8b2e06-52c1-4324-893d-dfe08699a6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:28.215729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1465/2920] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:10:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:10:28.236425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1ae51-ee0d-459b-8aaa-d6f45b7ed72b admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:28.238753 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-cca1ae51-ee0d-459b-8aaa-d6f45b7ed72b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:28.360661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1ae51-ee0d-459b-8aaa-d6f45b7ed72b admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:28.360661 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e505b-a79e-4b2f-be53-59fa4d29da00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:28.361380 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e505b-a79e-4b2f-be53-59fa4d29da00 service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:28.361380 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e505b-a79e-4b2f-be53-59fa4d29da00 service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:28.361380 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e505b-a79e-4b2f-be53-59fa4d29da00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:28.426232 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e505b-a79e-4b2f-be53-59fa4d29da00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:28.428746 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e505b-a79e-4b2f-be53-59fa4d29da00 service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:28.439559 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1456/2921] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:10:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:10:38.203883 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=67 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:10:38.435838 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=66,keystone:INSERT=2 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:10:43.731504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad82cf-444e-48c6-9550-dda22794929a None None] REQUEST_METHOD: `POST` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:43.732048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad82cf-444e-48c6-9550-dda22794929a None None] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:43.732048 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad82cf-444e-48c6-9550-dda22794929a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:43.751276 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.auth.core [None req-26ad82cf-444e-48c6-9550-dda22794929a None None] MFA Rules not processed for user `8d49d3422c7d420db80fe5814a768f53`. Rule list: `[]` (Enabled: `True`). {{(pid=81454) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:10:43.752475 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-26ad82cf-444e-48c6-9550-dda22794929a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:43.759910 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1466/2922] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:10:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:10:53.745756 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=3 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:10:56.284358 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e505b-a79e-4b2f-be53-59fa4d29da00 service nova] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:56.296254 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e505b-a79e-4b2f-be53-59fa4d29da00 service nova] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:56.302116 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecebd71-186b-4d99-8bb6-93a4adebb7db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:56.303258 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecebd71-186b-4d99-8bb6-93a4adebb7db service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:56.303538 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecebd71-186b-4d99-8bb6-93a4adebb7db service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:56.303810 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecebd71-186b-4d99-8bb6-93a4adebb7db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:56.307589 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-4ecebd71-186b-4d99-8bb6-93a4adebb7db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:56.321523 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecebd71-186b-4d99-8bb6-93a4adebb7db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:56.323783 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecebd71-186b-4d99-8bb6-93a4adebb7db service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:56.335288 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1457/2923] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:10:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:10:58.833614 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad82cf-444e-48c6-9550-dda22794929a None None] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:10:58.835661 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-26ad82cf-444e-48c6-9550-dda22794929a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:10:58.893502 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad82cf-444e-48c6-9550-dda22794929a None None] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:10:58.898284 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a44666-949f-4ad8-a6f1-0b2bcc721d48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:10:58.898933 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a44666-949f-4ad8-a6f1-0b2bcc721d48 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:10:58.899206 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a44666-949f-4ad8-a6f1-0b2bcc721d48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:10:58.899470 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a44666-949f-4ad8-a6f1-0b2bcc721d48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:10:58.909607 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a44666-949f-4ad8-a6f1-0b2bcc721d48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:10:58.911283 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a44666-949f-4ad8-a6f1-0b2bcc721d48 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:10:58.920965 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1467/2924] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:10:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:11:00.473927 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecebd71-186b-4d99-8bb6-93a4adebb7db service nova] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:11:00.485059 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecebd71-186b-4d99-8bb6-93a4adebb7db service nova] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:11:00.490498 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12becca5-152e-46b5-9b65-977ddb744507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:11:00.491135 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12becca5-152e-46b5-9b65-977ddb744507 service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:11:00.491413 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12becca5-152e-46b5-9b65-977ddb744507 service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:11:00.491683 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12becca5-152e-46b5-9b65-977ddb744507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:11:00.493425 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-12becca5-152e-46b5-9b65-977ddb744507 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:11:00.506468 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12becca5-152e-46b5-9b65-977ddb744507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:11:00.508052 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12becca5-152e-46b5-9b65-977ddb744507 service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:11:00.519039 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1458/2925] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:11:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:11:08.916637 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=26 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:11:10.514059 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=12 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:11:20.623731 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a44666-949f-4ad8-a6f1-0b2bcc721d48 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:11:20.625180 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-f9a44666-949f-4ad8-a6f1-0b2bcc721d48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:11:20.692994 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a44666-949f-4ad8-a6f1-0b2bcc721d48 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:11:20.698606 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6a086-0909-4423-861f-8b62aef80733 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b6b34b4ee5492fb1f0699716e5118b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:11:20.699240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6a086-0909-4423-861f-8b62aef80733 service placement] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:11:20.699517 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6a086-0909-4423-861f-8b62aef80733 service placement] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:11:20.699787 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6a086-0909-4423-861f-8b62aef80733 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:11:20.701515 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-4ec6a086-0909-4423-861f-8b62aef80733 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:11:20.713606 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6a086-0909-4423-861f-8b62aef80733 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:11:20.715203 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6a086-0909-4423-861f-8b62aef80733 service placement] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:11:20.725262 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1468/2926] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:11:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:11:30.721118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=32 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:12:59.417094 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a751b7-d1b2-40d9-84d3-82c0baf8e266 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:12:59.417665 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a751b7-d1b2-40d9-84d3-82c0baf8e266 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:12:59.417772 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a751b7-d1b2-40d9-84d3-82c0baf8e266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:12:59.437016 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-37a751b7-d1b2-40d9-84d3-82c0baf8e266 None None] MFA Rules not processed for user `8d49d3422c7d420db80fe5814a768f53`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:12:59.438351 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-37a751b7-d1b2-40d9-84d3-82c0baf8e266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:12:59.446119 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1459/2927] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:12:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 15:13:09.431357 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=3 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:14:20.359954 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6a086-0909-4423-861f-8b62aef80733 service placement] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:14:20.369797 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6a086-0909-4423-861f-8b62aef80733 service placement] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:14:20.373306 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f3096-4ea6-4c62-b1bf-cd7f9b270e3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:14:20.373896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f3096-4ea6-4c62-b1bf-cd7f9b270e3f service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:14:20.374166 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f3096-4ea6-4c62-b1bf-cd7f9b270e3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:14:20.374432 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f3096-4ea6-4c62-b1bf-cd7f9b270e3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:14:20.376229 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-437f3096-4ea6-4c62-b1bf-cd7f9b270e3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:14:20.386240 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f3096-4ea6-4c62-b1bf-cd7f9b270e3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:14:20.387685 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f3096-4ea6-4c62-b1bf-cd7f9b270e3f service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:14:20.395508 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1469/2928] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:14:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:14:30.391959 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=6 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:14:44.358387 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25a612-c0e3-49bd-93e2-5e2750a868a0 None None] REQUEST_METHOD: `POST` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:14:44.358972 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25a612-c0e3-49bd-93e2-5e2750a868a0 None None] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:14:44.358972 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25a612-c0e3-49bd-93e2-5e2750a868a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:14:44.377570 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.auth.core [None req-cb25a612-c0e3-49bd-93e2-5e2750a868a0 None None] MFA Rules not processed for user `8d49d3422c7d420db80fe5814a768f53`. Rule list: `[]` (Enabled: `True`). {{(pid=81455) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 15:14:44.378837 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-cb25a612-c0e3-49bd-93e2-5e2750a868a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:14:44.386337 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1460/2929] 10.222.0.26 () {58 vars in 958 bytes} [Thu Sep 22 15:14:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 15:14:44.429575 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f3096-4ea6-4c62-b1bf-cd7f9b270e3f service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:14:44.431018 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-437f3096-4ea6-4c62-b1bf-cd7f9b270e3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:14:44.486026 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f3096-4ea6-4c62-b1bf-cd7f9b270e3f service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:14:44.489602 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3f537-aa0d-43db-9855-acb53757a6cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d265d111b534944b7812cf067e317a2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:14:44.490157 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3f537-aa0d-43db-9855-acb53757a6cc service cinder] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:14:44.490431 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3f537-aa0d-43db-9855-acb53757a6cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:14:44.490697 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3f537-aa0d-43db-9855-acb53757a6cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:14:44.492405 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-bbd3f537-aa0d-43db-9855-acb53757a6cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:14:44.505519 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3f537-aa0d-43db-9855-acb53757a6cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:14:44.506902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3f537-aa0d-43db-9855-acb53757a6cc service cinder] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:14:44.518064 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1470/2930] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:14:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:14:44.811365 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25a612-c0e3-49bd-93e2-5e2750a868a0 None None] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:14:44.821721 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25a612-c0e3-49bd-93e2-5e2750a868a0 None None] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:14:44.825265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2842e83-7422-4da1-9ef5-54e3de21394e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:14:44.825860 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2842e83-7422-4da1-9ef5-54e3de21394e service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:14:44.826147 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2842e83-7422-4da1-9ef5-54e3de21394e service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:14:44.826416 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2842e83-7422-4da1-9ef5-54e3de21394e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:14:44.828202 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-d2842e83-7422-4da1-9ef5-54e3de21394e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:14:44.837634 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2842e83-7422-4da1-9ef5-54e3de21394e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:14:44.839076 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2842e83-7422-4da1-9ef5-54e3de21394e service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:14:44.847038 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1461/2931] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:14:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:14:45.293056 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3f537-aa0d-43db-9855-acb53757a6cc service cinder] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:14:45.294589 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-bbd3f537-aa0d-43db-9855-acb53757a6cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:14:45.309541 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3f537-aa0d-43db-9855-acb53757a6cc service cinder] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:14:45.312070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2842e83-7422-4da1-9ef5-54e3de21394e service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:14:45.316083 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7dd1f-a343-4bcc-9ef7-9214df7887a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b6b34b4ee5492fb1f0699716e5118b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:14:45.316729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7dd1f-a343-4bcc-9ef7-9214df7887a3 service placement] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:14:45.317006 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7dd1f-a343-4bcc-9ef7-9214df7887a3 service placement] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:14:45.317278 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7dd1f-a343-4bcc-9ef7-9214df7887a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:14:45.319057 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-70b7dd1f-a343-4bcc-9ef7-9214df7887a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:14:45.320975 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2842e83-7422-4da1-9ef5-54e3de21394e service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:14:45.324675 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b565dc-d66c-40f0-ab6f-4b29197a269b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:14:45.325357 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b565dc-d66c-40f0-ab6f-4b29197a269b service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:14:45.325712 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b565dc-d66c-40f0-ab6f-4b29197a269b service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:14:45.326051 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b565dc-d66c-40f0-ab6f-4b29197a269b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:14:45.327896 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-20b565dc-d66c-40f0-ab6f-4b29197a269b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:14:45.331320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b7dd1f-a343-4bcc-9ef7-9214df7887a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:14:45.332942 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b7dd1f-a343-4bcc-9ef7-9214df7887a3 service placement] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:14:45.342199 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b565dc-d66c-40f0-ab6f-4b29197a269b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:14:45.343385 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1471/2932] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:14:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:14:45.343805 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b565dc-d66c-40f0-ab6f-4b29197a269b service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:14:45.356420 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1462/2933] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:14:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:14:55.339640 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=32 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:14:55.350354 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=15 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:16:23.220098 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7dd1f-a343-4bcc-9ef7-9214df7887a3 service placement] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:23.277706 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7dd1f-a343-4bcc-9ef7-9214df7887a3 service placement] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:23.283141 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e6688-0d51-462c-90d1-18f7519a2d7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b6b34b4ee5492fb1f0699716e5118b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:23.283749 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e6688-0d51-462c-90d1-18f7519a2d7f service placement] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:23.284026 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e6688-0d51-462c-90d1-18f7519a2d7f service placement] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:23.284300 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e6688-0d51-462c-90d1-18f7519a2d7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:23.293900 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e6688-0d51-462c-90d1-18f7519a2d7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:23.295531 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e6688-0d51-462c-90d1-18f7519a2d7f service placement] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:23.305395 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1472/2934] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:16:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:16:28.682161 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b565dc-d66c-40f0-ab6f-4b29197a269b service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:28.718782 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b565dc-d66c-40f0-ab6f-4b29197a269b service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:28.722680 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460a398-1798-4c60-a2ea-e7c72af7d3f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:28.723296 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460a398-1798-4c60-a2ea-e7c72af7d3f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:28.723571 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460a398-1798-4c60-a2ea-e7c72af7d3f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:28.723841 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460a398-1798-4c60-a2ea-e7c72af7d3f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:28.732265 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460a398-1798-4c60-a2ea-e7c72af7d3f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:28.733667 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460a398-1798-4c60-a2ea-e7c72af7d3f9 service neutron] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:28.742214 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1463/2935] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:16:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:16:29.582592 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e6688-0d51-462c-90d1-18f7519a2d7f service placement] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:29.598111 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e6688-0d51-462c-90d1-18f7519a2d7f service placement] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:29.607005 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48223a71-debd-42a8-ae6d-8cbf33f56a8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:29.607619 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48223a71-debd-42a8-ae6d-8cbf33f56a8b service nova] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:29.607896 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48223a71-debd-42a8-ae6d-8cbf33f56a8b service nova] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:29.608165 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48223a71-debd-42a8-ae6d-8cbf33f56a8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:29.618824 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48223a71-debd-42a8-ae6d-8cbf33f56a8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:29.620499 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48223a71-debd-42a8-ae6d-8cbf33f56a8b service nova] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:29.633160 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1473/2936] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:16:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:16:31.782938 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460a398-1798-4c60-a2ea-e7c72af7d3f9 service neutron] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:31.796485 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460a398-1798-4c60-a2ea-e7c72af7d3f9 service neutron] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:31.802027 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7809252-0c29-4454-8bbb-e55a0262d09d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c9be41c39494e54a14398a1f03c4d69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:31.802651 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7809252-0c29-4454-8bbb-e55a0262d09d service nova] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:31.802965 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7809252-0c29-4454-8bbb-e55a0262d09d service nova] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:31.803241 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7809252-0c29-4454-8bbb-e55a0262d09d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:31.805007 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-a7809252-0c29-4454-8bbb-e55a0262d09d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:31.829943 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7809252-0c29-4454-8bbb-e55a0262d09d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:31.831573 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7809252-0c29-4454-8bbb-e55a0262d09d service nova] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:31.842605 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1464/2937] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:16:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:16:39.628753 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=33 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:16:41.837909 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=28 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:16:41.859966 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48223a71-debd-42a8-ae6d-8cbf33f56a8b service nova] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:41.871256 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48223a71-debd-42a8-ae6d-8cbf33f56a8b service nova] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:41.874979 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a183d9d-1a8c-46b6-bfc9-c86308518f23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f77bbd8de87042aab3674db366635cf5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:41.875608 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a183d9d-1a8c-46b6-bfc9-c86308518f23 service neutron] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:41.875857 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a183d9d-1a8c-46b6-bfc9-c86308518f23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:41.876122 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a183d9d-1a8c-46b6-bfc9-c86308518f23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:41.949215 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a183d9d-1a8c-46b6-bfc9-c86308518f23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:41.950646 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a183d9d-1a8c-46b6-bfc9-c86308518f23 service neutron] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:41.968972 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1474/2938] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:16:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 15:16:45.056470 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7809252-0c29-4454-8bbb-e55a0262d09d service nova] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:45.077495 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7809252-0c29-4454-8bbb-e55a0262d09d service nova] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:45.084125 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9a5e4-119d-4b7a-8050-8432d4720d6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b6b34b4ee5492fb1f0699716e5118b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:45.085030 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9a5e4-119d-4b7a-8050-8432d4720d6e service placement] REQUEST_METHOD: `GET` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:45.085307 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9a5e4-119d-4b7a-8050-8432d4720d6e service placement] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:45.085587 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9a5e4-119d-4b7a-8050-8432d4720d6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:45.088222 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-def9a5e4-119d-4b7a-8050-8432d4720d6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:45.121468 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9a5e4-119d-4b7a-8050-8432d4720d6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:45.125393 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9a5e4-119d-4b7a-8050-8432d4720d6e service placement] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:45.144803 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1465/2939] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:16:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:16:48.298704 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a183d9d-1a8c-46b6-bfc9-c86308518f23 service neutron] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:48.311118 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a183d9d-1a8c-46b6-bfc9-c86308518f23 service neutron] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:48.316119 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80938ab-0a4c-4933-9150-8b3bc1dfebcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:48.316785 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80938ab-0a4c-4933-9150-8b3bc1dfebcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:48.317062 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80938ab-0a4c-4933-9150-8b3bc1dfebcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:48.317330 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80938ab-0a4c-4933-9150-8b3bc1dfebcf admin admin] PATH_INFO: `/v3/users/8d49d3422c7d420db80fe5814a768f53` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:48.318504 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80938ab-0a4c-4933-9150-8b3bc1dfebcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d49d3422c7d420db80fe5814a768f53)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:48.320503 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80938ab-0a4c-4933-9150-8b3bc1dfebcf admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:48.374091 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a80938ab-0a4c-4933-9150-8b3bc1dfebcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d49d3422c7d420db80fe5814a768f53'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:48.382409 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a80938ab-0a4c-4933-9150-8b3bc1dfebcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d49d3422c7d420db80fe5814a768f53'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:48.392355 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a80938ab-0a4c-4933-9150-8b3bc1dfebcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d49d3422c7d420db80fe5814a768f53'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:48.406053 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a80938ab-0a4c-4933-9150-8b3bc1dfebcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d49d3422c7d420db80fe5814a768f53'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:48.406808 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-a80938ab-0a4c-4933-9150-8b3bc1dfebcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d49d3422c7d420db80fe5814a768f53'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:48.412585 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1475/2940] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:16:48 2022] DELETE /identity/v3/users/8d49d3422c7d420db80fe5814a768f53 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:16:48.595736 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9a5e4-119d-4b7a-8050-8432d4720d6e service placement] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:48.611394 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9a5e4-119d-4b7a-8050-8432d4720d6e service placement] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:48.673745 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb025dbf-1728-425f-a0cf-9a99b6a87162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:48.674385 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb025dbf-1728-425f-a0cf-9a99b6a87162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:48.674660 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb025dbf-1728-425f-a0cf-9a99b6a87162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:48.674963 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb025dbf-1728-425f-a0cf-9a99b6a87162 admin admin] PATH_INFO: `/v3/projects/1b64f192d3ea436abc310162b1c46efc` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:48.676095 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb025dbf-1728-425f-a0cf-9a99b6a87162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b64f192d3ea436abc310162b1c46efc)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:48.678043 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb025dbf-1728-425f-a0cf-9a99b6a87162 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:48.730476 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-eb025dbf-1728-425f-a0cf-9a99b6a87162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b64f192d3ea436abc310162b1c46efc', 'request_id': 'req-eb025dbf-1728-425f-a0cf-9a99b6a87162'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:48.738452 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-eb025dbf-1728-425f-a0cf-9a99b6a87162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b64f192d3ea436abc310162b1c46efc', 'request_id': 'req-eb025dbf-1728-425f-a0cf-9a99b6a87162'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:48.746070 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-eb025dbf-1728-425f-a0cf-9a99b6a87162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b64f192d3ea436abc310162b1c46efc', 'request_id': 'req-eb025dbf-1728-425f-a0cf-9a99b6a87162'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:48.770183 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-eb025dbf-1728-425f-a0cf-9a99b6a87162 admin admin] The token cache is being invalidate because project 1b64f192d3ea436abc310162b1c46efc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:16:48.770408 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-eb025dbf-1728-425f-a0cf-9a99b6a87162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:48.771045 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-eb025dbf-1728-425f-a0cf-9a99b6a87162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:48.772550 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1466/2941] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:16:48 2022] DELETE /identity/v3/projects/1b64f192d3ea436abc310162b1c46efc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:16:48.776794 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80938ab-0a4c-4933-9150-8b3bc1dfebcf admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:48.778320 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-a80938ab-0a4c-4933-9150-8b3bc1dfebcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:48.791682 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80938ab-0a4c-4933-9150-8b3bc1dfebcf admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:48.837902 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:16:48.851804 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a27c73-9eac-4c15-920c-ebcd7016194f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:48.852445 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a27c73-9eac-4c15-920c-ebcd7016194f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:48.852725 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a27c73-9eac-4c15-920c-ebcd7016194f admin admin] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:48.852999 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a27c73-9eac-4c15-920c-ebcd7016194f admin admin] PATH_INFO: `/v3/users/76d2d9b7116e46b0a93b5aabd019018d` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:48.854140 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a27c73-9eac-4c15-920c-ebcd7016194f admin admin] RBAC: Authorizing `identity:delete_user(user_id=76d2d9b7116e46b0a93b5aabd019018d)` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:48.856675 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a27c73-9eac-4c15-920c-ebcd7016194f admin admin] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:48.904976 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b8a27c73-9eac-4c15-920c-ebcd7016194f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76d2d9b7116e46b0a93b5aabd019018d'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:48.912192 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b8a27c73-9eac-4c15-920c-ebcd7016194f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76d2d9b7116e46b0a93b5aabd019018d'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:48.920132 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b8a27c73-9eac-4c15-920c-ebcd7016194f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76d2d9b7116e46b0a93b5aabd019018d'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:48.932031 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b8a27c73-9eac-4c15-920c-ebcd7016194f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76d2d9b7116e46b0a93b5aabd019018d'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:48.933281 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.notifications [None req-b8a27c73-9eac-4c15-920c-ebcd7016194f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76d2d9b7116e46b0a93b5aabd019018d'} {{(pid=81454) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:48.939238 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1476/2942] 10.222.0.26 () {58 vars in 1224 bytes} [Thu Sep 22 15:16:48 2022] DELETE /identity/v3/users/76d2d9b7116e46b0a93b5aabd019018d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:16:49.126846 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb025dbf-1728-425f-a0cf-9a99b6a87162 admin admin] Authenticating user token {{(pid=81455) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:16:49.130293 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.fernet_utils [None req-eb025dbf-1728-425f-a0cf-9a99b6a87162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81455) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:16:49.145032 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb025dbf-1728-425f-a0cf-9a99b6a87162 admin admin] Validating token access rules against request {{(pid=81455) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:16:49.203378 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:16:49.212023 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739d3ea-4f80-4c9d-bdbc-03bae460f194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c56fbc6ee754a96af39128dae8f43a4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82e4fd5adbe54235a872491a4adc6897', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81455) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:16:49.212827 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739d3ea-4f80-4c9d-bdbc-03bae460f194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:16:49.213468 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739d3ea-4f80-4c9d-bdbc-03bae460f194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:16:49.214213 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739d3ea-4f80-4c9d-bdbc-03bae460f194 admin admin] PATH_INFO: `/v3/projects/8f9e9a78f2fc497787dc8981cc54e770` {{(pid=81455) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:16:49.215675 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739d3ea-4f80-4c9d-bdbc-03bae460f194 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f9e9a78f2fc497787dc8981cc54e770)` {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:16:49.217926 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739d3ea-4f80-4c9d-bdbc-03bae460f194 admin admin] RBAC: Authorization granted {{(pid=81455) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:16:49.272872 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b739d3ea-4f80-4c9d-bdbc-03bae460f194 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f9e9a78f2fc497787dc8981cc54e770', 'request_id': 'req-b739d3ea-4f80-4c9d-bdbc-03bae460f194'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:49.281515 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b739d3ea-4f80-4c9d-bdbc-03bae460f194 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f9e9a78f2fc497787dc8981cc54e770', 'request_id': 'req-b739d3ea-4f80-4c9d-bdbc-03bae460f194'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:49.289421 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b739d3ea-4f80-4c9d-bdbc-03bae460f194 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f9e9a78f2fc497787dc8981cc54e770', 'request_id': 'req-b739d3ea-4f80-4c9d-bdbc-03bae460f194'} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:49.302459 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b739d3ea-4f80-4c9d-bdbc-03bae460f194 admin admin] The token cache is being invalidate because project 8f9e9a78f2fc497787dc8981cc54e770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81455) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 15:16:49.302833 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b739d3ea-4f80-4c9d-bdbc-03bae460f194 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:49.303502 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG keystone.notifications [None req-b739d3ea-4f80-4c9d-bdbc-03bae460f194 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81455) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 15:16:49.305321 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: [pid: 81455|app: 0|req: 1467/2943] 10.222.0.26 () {58 vars in 1233 bytes} [Thu Sep 22 15:16:49 2022] DELETE /identity/v3/projects/8f9e9a78f2fc497787dc8981cc54e770 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 15:16:58.935693 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:16:59.298300 dv-cinder-844563-6-iscsi devstack@keystone.service[81455]: DEBUG dbcounter [-] [81455] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81455) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 15:21:25.596628 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a27c73-9eac-4c15-920c-ebcd7016194f admin admin] Authenticating user token {{(pid=81454) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 15:21:25.598080 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-b8a27c73-9eac-4c15-920c-ebcd7016194f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:21:25.669816 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a27c73-9eac-4c15-920c-ebcd7016194f admin admin] Validating token access rules against request {{(pid=81454) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 15:21:25.734716 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4ba06-3abf-439f-a082-07a84c2edce5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47b6b34b4ee5492fb1f0699716e5118b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '43bc5871e4bf441b874474bcae4e5385', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81454) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 15:21:25.735353 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4ba06-3abf-439f-a082-07a84c2edce5 service placement] REQUEST_METHOD: `GET` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 15:21:25.735632 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4ba06-3abf-439f-a082-07a84c2edce5 service placement] SCRIPT_NAME: `/identity` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 15:21:25.735903 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4ba06-3abf-439f-a082-07a84c2edce5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81454) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 15:21:25.737653 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.fernet_utils [None req-30f4ba06-3abf-439f-a082-07a84c2edce5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81454) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 15:21:25.748841 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4ba06-3abf-439f-a082-07a84c2edce5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 15:21:25.750413 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f4ba06-3abf-439f-a082-07a84c2edce5 service placement] RBAC: Authorization granted {{(pid=81454) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 15:21:25.760729 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: [pid: 81454|app: 0|req: 1477/2944] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Sep 22 15:21:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 15:21:35.756533 dv-cinder-844563-6-iscsi devstack@keystone.service[81454]: DEBUG dbcounter [-] [81454] Writing DB stats keystone:SELECT=58 {{(pid=81454) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}